Oracle Database 11g CRS Release 11.1.0.7.0 - Production Copyright 1996, 2007 Oracle. All rights reserved. [ CSSD]2013-05-23 15:34:23.328 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 606 (0x7f431bfd1f10) [ CSSD]2013-05-23 15:34:23.328 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1f10 (RPC#606) tag(25e002c) sent to node 1 [ CSSD]2013-05-23 15:34:23.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(025e002c) incarnation(174631051) [ CSSD]2013-05-23 15:34:23.328 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2013997, gin 1, member 0 [ CSSD]2013-05-23 15:34:23.328 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec570/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:34:23.329 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:34:23.329 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:34:23.329 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1f10 (RPC#606) state 6, flags 0x100 [ CSSD]2013-05-23 15:34:23.329 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1f10, ret 0, client 0x22f5430 member 0x2318d80 [ CSSD]2013-05-23 15:34:23.329 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f5430 [ CSSD]2013-05-23 15:34:23.329 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 606 [ CSSD]2013-05-23 15:34:23.335 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f5430) [ CSSD]2013-05-23 15:34:23.335 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f5430), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:34:23.335 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:34:23.335 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 607 (0x7f431bfd1f90) [ CSSD]2013-05-23 15:34:23.335 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1f90 (RPC#607) tag(25f002c) sent to node 1 [ CSSD]2013-05-23 15:34:23.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(025f002c) incarnation(174631051) [ CSSD]2013-05-23 15:34:23.335 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:34:23.335 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1f90 (RPC#607) state 6, flags 0x100 [ CSSD]2013-05-23 15:34:23.335 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1f90, ret 0, client 0x22f5430 member 0x2318d80 [ CSSD]2013-05-23 15:34:23.336 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 607 [ CSSD]2013-05-23 15:34:23.336 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:34:23.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:23.375 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:23.538 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e80 [ CSSD]2013-05-23 15:34:23.538 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e80) con(0x22f4610) skgpid ospid 8727 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:34:25.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:25.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:25.394 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:25.636 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:25.636 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:27.142 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:27.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:27.400 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:29.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:29.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:29.411 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:30.639 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:30.639 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:31.151 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:31.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:31.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:33.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:33.274 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:34:33.274 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 608 (0x7f431bfd2010) [ CSSD]2013-05-23 15:34:33.274 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2010 (RPC#608) tag(260002c) sent to node 1 [ CSSD]2013-05-23 15:34:33.275 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:33.275 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 608 (0x7f431bfd2010) [ CSSD]2013-05-23 15:34:33.275 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2010 (RPC#608) state 6, flags 0x202 [ CSSD]2013-05-23 15:34:33.275 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2010, ret 0, client (nil) [ CSSD]2013-05-23 15:34:33.275 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 608 [ CSSD]2013-05-23 15:34:33.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:33.469 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:33.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:33.818 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2013998, gin 1, member 0 [ CSSD]2013-05-23 15:34:33.818 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:34:33.818 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:34:33.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:33.824 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:34:33.824 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:34:35.171 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:35.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:35.475 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:35.642 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:35.642 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:36.126 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22d7970), client(2/0x7f431c12a550) [ CSSD]2013-05-23 15:34:36.126 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c12a550) [ CSSD]2013-05-23 15:34:36.126 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:34:36.126 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 15:34:37.182 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:37.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:37.480 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:38.581 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0cf650) allocated [ CSSD]2013-05-23 15:34:38.582 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:34:38.582 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0cf650 - 1,2,3 [ CSSD]2013-05-23 15:34:38.582 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c0cf650) pid(8869) version 11:1:1:4 [ CSSD]2013-05-23 15:34:38.588 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0cf650), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 15:34:38.588 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:34:38.588 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 15:34:38.588 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:34:38.588 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 609 (0x7f431bfd2090) [ CSSD]2013-05-23 15:34:38.588 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2090 (RPC#609) tag(261002c) sent to node 1 [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0261002c) incarnation(174631051) [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2013999, gin 1, member 0 [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2090 (RPC#609) state 6, flags 0x100 [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2090, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 15:34:38.589 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 609 [ CSSD]2013-05-23 15:34:38.595 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:34:38.595 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:34:38.595 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:34:38.595 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 610 (0x7f431bfd2110) [ CSSD]2013-05-23 15:34:38.595 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2110 (RPC#610) tag(262002c) sent to node 1 [ CSSD]2013-05-23 15:34:38.596 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0262002c) incarnation(174631051) [ CSSD]2013-05-23 15:34:38.596 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:34:38.596 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2110 (RPC#610) state 6, flags 0x100 [ CSSD]2013-05-23 15:34:38.596 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2110, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:34:38.596 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 610 [ CSSD]2013-05-23 15:34:38.596 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:34:38.798 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0cf650 [ CSSD]2013-05-23 15:34:38.798 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0cf650) con(0x7f431c0e6720) skgpid ospid 8869 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:34:39.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:39.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:39.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:40.645 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:40.645 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:41.194 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:41.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:41.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:43.245 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:43.315 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:34:43.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 611 (0x7f431bfd2190) [ CSSD]2013-05-23 15:34:43.315 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2190 (RPC#611) tag(263002c) sent to node 1 [ CSSD]2013-05-23 15:34:43.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:43.316 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 611 (0x7f431bfd2190) [ CSSD]2013-05-23 15:34:43.316 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2190 (RPC#611) state 6, flags 0x202 [ CSSD]2013-05-23 15:34:43.316 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2190, ret 0, client (nil) [ CSSD]2013-05-23 15:34:43.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 611 [ CSSD]2013-05-23 15:34:43.569 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:43.569 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:45.290 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:45.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:45.594 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:45.648 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:45.648 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:47.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:47.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:47.600 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:49.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:49.069 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014000, gin 1, member 0 [ CSSD]2013-05-23 15:34:49.069 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:34:49.069 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:34:49.078 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:49.078 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:34:49.078 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:34:49.348 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:49.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:49.610 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:50.651 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:50.651 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:51.354 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:51.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:51.616 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:53.345 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:34:53.345 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 612 (0x7f431bfd2210) [ CSSD]2013-05-23 15:34:53.345 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2210 (RPC#612) tag(264002c) sent to node 1 [ CSSD]2013-05-23 15:34:53.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:53.346 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 612 (0x7f431bfd2210) [ CSSD]2013-05-23 15:34:53.346 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2210 (RPC#612) state 6, flags 0x202 [ CSSD]2013-05-23 15:34:53.346 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2210, ret 0, client (nil) [ CSSD]2013-05-23 15:34:53.346 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 612 [ CSSD]2013-05-23 15:34:53.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:53.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:53.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:53.831 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0cf650) allocated [ CSSD]2013-05-23 15:34:53.832 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:34:53.832 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0cf650 - 1,2,3 [ CSSD]2013-05-23 15:34:53.832 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c0cf650) pid(9030) version 11:1:1:4 [ CSSD]2013-05-23 15:34:53.848 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0cf650), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 15:34:53.848 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:34:53.848 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 15:34:53.848 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:34:53.848 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 613 (0x7f431bfd2290) [ CSSD]2013-05-23 15:34:53.849 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2290 (RPC#613) tag(265002c) sent to node 1 [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0265002c) incarnation(174631051) [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014001, gin 1, member 0 [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2290 (RPC#613) state 6, flags 0x100 [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2290, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:34:53.849 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 15:34:53.850 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 613 [ CSSD]2013-05-23 15:34:53.856 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:34:53.856 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:34:53.856 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:34:53.856 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 614 (0x7f431bfd2310) [ CSSD]2013-05-23 15:34:53.856 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2310 (RPC#614) tag(266002c) sent to node 1 [ CSSD]2013-05-23 15:34:53.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0266002c) incarnation(174631051) [ CSSD]2013-05-23 15:34:53.856 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:34:53.856 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2310 (RPC#614) state 6, flags 0x100 [ CSSD]2013-05-23 15:34:53.856 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2310, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:34:53.856 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 614 [ CSSD]2013-05-23 15:34:53.856 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:34:54.059 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0cf650 [ CSSD]2013-05-23 15:34:54.059 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0cf650) con(0x7f431c0e6720) skgpid ospid 9030 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:34:55.390 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:55.654 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:34:55.654 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:34:55.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:55.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:57.443 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:57.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:57.680 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:34:59.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:34:59.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:34:59.691 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:00.657 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:00.657 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:01.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:01.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:01.717 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:03.375 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:35:03.376 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 615 (0x7f431bfd2390) [ CSSD]2013-05-23 15:35:03.376 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2390 (RPC#615) tag(267002c) sent to node 1 [ CSSD]2013-05-23 15:35:03.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:03.376 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 615 (0x7f431bfd2390) [ CSSD]2013-05-23 15:35:03.376 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2390 (RPC#615) state 6, flags 0x202 [ CSSD]2013-05-23 15:35:03.376 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2390, ret 0, client (nil) [ CSSD]2013-05-23 15:35:03.376 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 615 [ CSSD]2013-05-23 15:35:03.466 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:03.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:03.553 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014002, gin 1, member 0 [ CSSD]2013-05-23 15:35:03.553 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:35:03.553 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:35:03.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:03.560 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:35:03.560 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:35:03.720 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:03.720 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:04.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:04.324 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014003, gin 1, member 0 [ CSSD]2013-05-23 15:35:04.324 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:35:04.324 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:35:04.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:04.330 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:35:04.330 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:35:04.660 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:04.660 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:35:05.471 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:05.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:05.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:07.301 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 15:35:07.302 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:35:07.302 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 15:35:07.302 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108da0) pid(9255) version 11:1:1:4 [ CSSD]2013-05-23 15:35:07.307 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108da0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 15:35:07.307 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:35:07.307 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:35:07.307 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:07.307 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 616 (0x7f431bfd2410) [ CSSD]2013-05-23 15:35:07.307 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2410 (RPC#616) tag(268002c) sent to node 1 [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0268002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014004, gin 1, member 0 [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2410 (RPC#616) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2410, ret 0, client 0x7f431c0e6e30 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 15:35:07.308 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 616 [ CSSD]2013-05-23 15:35:07.314 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:35:07.314 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:07.314 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:07.314 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 617 (0x7f431bfd2490) [ CSSD]2013-05-23 15:35:07.314 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2490 (RPC#617) tag(269002c) sent to node 1 [ CSSD]2013-05-23 15:35:07.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0269002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:07.315 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:07.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2490 (RPC#617) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:07.315 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2490, ret 0, client 0x7f431c0e6e30 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:35:07.315 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 617 [ CSSD]2013-05-23 15:35:07.315 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:35:07.517 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 15:35:07.517 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c0e6720) skgpid ospid 9255 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:35:07.544 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:07.652 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e6d60) allocated [ CSSD]2013-05-23 15:35:07.653 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:35:07.653 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e6d60 - 1,2,3 [ CSSD]2013-05-23 15:35:07.653 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0e6d60) pid(9236) version 11:1:1:4 [ CSSD]2013-05-23 15:35:07.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:07.730 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:08.137 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e6d60 [ CSSD]2013-05-23 15:35:08.137 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e6d60) con(0x7f431c12f0c0) skgpid ospid 9236 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:35:09.089 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135050) allocated [ CSSD]2013-05-23 15:35:09.090 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:35:09.091 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135050 - 1,2,3 [ CSSD]2013-05-23 15:35:09.091 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c135050) pid(9354) version 11:1:1:4 [ CSSD]2013-05-23 15:35:09.097 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c135050), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:35:09.097 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:35:09.097 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:35:09.097 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:09.097 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 618 (0x7f431bfd2510) [ CSSD]2013-05-23 15:35:09.097 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2510 (RPC#618) tag(26a002c) sent to node 1 [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(026a002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014005, gin 1, member 0 [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2510 (RPC#618) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2510, ret 0, client 0x7f431c10a560 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:35:09.098 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 618 [ CSSD]2013-05-23 15:35:09.104 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:35:09.104 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:09.104 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:09.104 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 619 (0x7f431bfd2590) [ CSSD]2013-05-23 15:35:09.104 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2590 (RPC#619) tag(26b002c) sent to node 1 [ CSSD]2013-05-23 15:35:09.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(026b002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:09.104 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:09.104 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2590 (RPC#619) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:09.104 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2590, ret 0, client 0x7f431c10a560 member 0x7f431c11fd30 [ CSSD]2013-05-23 15:35:09.105 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 619 [ CSSD]2013-05-23 15:35:09.105 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:35:09.307 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135050 [ CSSD]2013-05-23 15:35:09.307 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135050) con(0x7f431c0e5200) skgpid ospid 9354 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:35:09.549 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:09.663 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:09.663 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:09.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:09.783 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:11.575 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:11.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:11.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:13.406 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:35:13.406 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 620 (0x7f431bfd2610) [ CSSD]2013-05-23 15:35:13.406 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2610 (RPC#620) tag(26c002c) sent to node 1 [ CSSD]2013-05-23 15:35:13.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:13.407 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 620 (0x7f431bfd2610) [ CSSD]2013-05-23 15:35:13.407 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2610 (RPC#620) state 6, flags 0x202 [ CSSD]2013-05-23 15:35:13.407 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2610, ret 0, client (nil) [ CSSD]2013-05-23 15:35:13.407 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 620 [ CSSD]2013-05-23 15:35:13.586 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:13.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:13.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:14.666 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:14.666 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:15.591 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:15.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:15.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:17.644 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:17.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:17.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:19.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:19.574 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014006, gin 1, member 0 [ CSSD]2013-05-23 15:35:19.574 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:35:19.574 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:35:19.580 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:19.580 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:35:19.580 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:35:19.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:19.669 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:19.669 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:19.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:19.823 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:21.654 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:21.829 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:21.829 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:22.305 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0cf650) allocated [ CSSD]2013-05-23 15:35:22.306 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:35:22.307 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0cf650 - 1,2,3 [ CSSD]2013-05-23 15:35:22.307 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c0cf650) pid(9483/9483) version 11:1:1:4 [ CSSD]2013-05-23 15:35:22.323 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0cf650), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 15:35:22.323 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:35:22.323 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:35:22.324 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 15:35:23.436 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:35:23.436 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 621 (0x7f431bfd2690) [ CSSD]2013-05-23 15:35:23.436 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2690 (RPC#621) tag(26d002c) sent to node 1 [ CSSD]2013-05-23 15:35:23.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:23.437 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 621 (0x7f431bfd2690) [ CSSD]2013-05-23 15:35:23.437 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2690 (RPC#621) state 6, flags 0x202 [ CSSD]2013-05-23 15:35:23.437 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2690, ret 0, client (nil) [ CSSD]2013-05-23 15:35:23.437 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 621 [ CSSD]2013-05-23 15:35:23.666 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:23.671 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:23.671 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:35:23.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:23.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:24.350 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 15:35:24.351 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:35:24.351 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 15:35:24.351 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(9540) version 11:1:1:4 [ CSSD]2013-05-23 15:35:24.367 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:35:24.367 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:35:24.367 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 15:35:24.367 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:24.367 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 622 (0x7f431bfd2710) [ CSSD]2013-05-23 15:35:24.368 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2710 (RPC#622) tag(26e002c) sent to node 1 [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(026e002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014007, gin 1, member 0 [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2710 (RPC#622) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2710, ret 0, client 0x7f431c135120 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:35:24.368 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 622 [ CSSD]2013-05-23 15:35:24.375 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:35:24.375 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:24.375 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1247a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:24.375 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 623 (0x7f431bfd2790) [ CSSD]2013-05-23 15:35:24.375 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2790 (RPC#623) tag(26f002c) sent to node 1 [ CSSD]2013-05-23 15:35:24.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(026f002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:24.375 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:24.375 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2790 (RPC#623) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:24.375 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2790, ret 0, client 0x7f431c135120 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:35:24.376 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 623 [ CSSD]2013-05-23 15:35:24.376 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1247a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:35:24.578 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 15:35:24.578 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 9540 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:35:25.671 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:25.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:25.886 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:27.745 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:27.892 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:27.892 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:28.674 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:28.675 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:28.932 [1191196992] >TRACE: clssgmAllocProc: (0x22f2de0) allocated [ CSSD]2013-05-23 15:35:28.932 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:35:28.933 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2de0 - 1,2,3 [ CSSD]2013-05-23 15:35:28.933 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x22f2de0) pid(8380/8380) version 11:1:1:4 [ CSSD]2013-05-23 15:35:28.939 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f2de0), client(1/0x22f4340) [ CSSD]2013-05-23 15:35:28.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f4340) [ CSSD]2013-05-23 15:35:28.939 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:35:28.939 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 15:35:29.770 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:29.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:29.904 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:31.776 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:31.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:31.909 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:33.466 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:35:33.466 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 624 (0x7f431bfd2810) [ CSSD]2013-05-23 15:35:33.467 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2810 (RPC#624) tag(270002c) sent to node 1 [ CSSD]2013-05-23 15:35:33.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:33.467 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 624 (0x7f431bfd2810) [ CSSD]2013-05-23 15:35:33.467 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2810 (RPC#624) state 6, flags 0x202 [ CSSD]2013-05-23 15:35:33.467 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2810, ret 0, client (nil) [ CSSD]2013-05-23 15:35:33.467 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 624 [ CSSD]2013-05-23 15:35:33.678 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:33.678 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:33.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:33.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:33.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:34.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:34.824 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014008, gin 1, member 0 [ CSSD]2013-05-23 15:35:34.824 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:35:34.824 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:35:34.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:34.831 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:35:34.831 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:35:35.794 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:35.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:35.926 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:37.845 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:37.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:37.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:38.681 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:38.681 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:39.610 [1191196992] >TRACE: clssgmAllocProc: (0x22edea0) allocated [ CSSD]2013-05-23 15:35:39.611 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:35:39.611 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edea0 - 1,2,3 [ CSSD]2013-05-23 15:35:39.611 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4150) proc(0x22edea0) pid(9699) version 11:1:1:4 [ CSSD]2013-05-23 15:35:39.617 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22edea0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:35:39.617 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:35:39.617 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x22ee100, requested num -1 [ CSSD]2013-05-23 15:35:39.617 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:39.617 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 625 (0x7f431bfd2890) [ CSSD]2013-05-23 15:35:39.618 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2890 (RPC#625) tag(271002c) sent to node 1 [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0271002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014009, gin 1, member 0 [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318710/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2890 (RPC#625) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2890, ret 0, client 0x22ee1b0 member 0x22edd90 [ CSSD]2013-05-23 15:35:39.618 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:35:39.619 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 625 [ CSSD]2013-05-23 15:35:39.624 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:35:39.625 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:39.625 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edd90) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:39.625 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 626 (0x7f431bfd2910) [ CSSD]2013-05-23 15:35:39.625 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2910 (RPC#626) tag(272002c) sent to node 1 [ CSSD]2013-05-23 15:35:39.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0272002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:39.625 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:39.625 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2910 (RPC#626) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:39.625 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2910, ret 0, client 0x22ee1b0 member 0x22edd90 [ CSSD]2013-05-23 15:35:39.625 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 626 [ CSSD]2013-05-23 15:35:39.625 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22edd90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:35:39.828 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edea0 [ CSSD]2013-05-23 15:35:39.828 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edea0) con(0x22f4150) skgpid ospid 9699 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:35:39.851 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:40.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:40.004 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:41.876 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:42.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:42.029 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:42.684 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:42.684 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:35:43.506 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:35:43.507 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 627 (0x7f431bfd2990) [ CSSD]2013-05-23 15:35:43.507 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2990 (RPC#627) tag(273002c) sent to node 1 [ CSSD]2013-05-23 15:35:43.507 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:43.507 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 627 (0x7f431bfd2990) [ CSSD]2013-05-23 15:35:43.507 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2990 (RPC#627) state 6, flags 0x202 [ CSSD]2013-05-23 15:35:43.507 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2990, ret 0, client (nil) [ CSSD]2013-05-23 15:35:43.507 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 627 [ CSSD]2013-05-23 15:35:43.887 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:44.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:44.081 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:45.893 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:46.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:46.106 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:47.687 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:47.687 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:47.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:48.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:48.112 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:49.280 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f2de0), client(2/0x22f4820) [ CSSD]2013-05-23 15:35:49.280 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f4820) [ CSSD]2013-05-23 15:35:49.280 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:35:49.280 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 15:35:49.921 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:50.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:50.074 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014010, gin 1, member 0 [ CSSD]2013-05-23 15:35:50.074 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:35:50.074 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:35:50.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:50.080 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:35:50.080 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:35:50.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:50.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:51.953 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:52.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:52.130 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:52.690 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:52.691 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:53.530 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:35:53.530 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 628 (0x7f431bfd2a10) [ CSSD]2013-05-23 15:35:53.530 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2a10 (RPC#628) tag(274002c) sent to node 1 [ CSSD]2013-05-23 15:35:53.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:53.531 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 628 (0x7f431bfd2a10) [ CSSD]2013-05-23 15:35:53.531 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2a10 (RPC#628) state 6, flags 0x202 [ CSSD]2013-05-23 15:35:53.531 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2a10, ret 0, client (nil) [ CSSD]2013-05-23 15:35:53.531 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 628 [ CSSD]2013-05-23 15:35:53.959 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:54.182 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:54.182 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:54.859 [1191196992] >TRACE: clssgmAllocProc: (0x23183f0) allocated [ CSSD]2013-05-23 15:35:54.860 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:35:54.861 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23183f0 - 1,2,3 [ CSSD]2013-05-23 15:35:54.861 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecae0) proc(0x23183f0) pid(9855) version 11:1:1:4 [ CSSD]2013-05-23 15:35:54.867 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x23183f0), client(1/0x2318b50) [ CSSD]2013-05-23 15:35:54.867 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318b50) [ CSSD]2013-05-23 15:35:54.867 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318b50 with con 0x2318650, requested num -1 [ CSSD]2013-05-23 15:35:54.867 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:54.867 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 629 (0x7f431bfd2a90) [ CSSD]2013-05-23 15:35:54.867 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2a90 (RPC#629) tag(275002c) sent to node 1 [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0275002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014011, gin 1, member 0 [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318cc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2a90 (RPC#629) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2a90, ret 0, client 0x2318b50 member 0x23182e0 [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318b50 [ CSSD]2013-05-23 15:35:54.868 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 629 [ CSSD]2013-05-23 15:35:54.875 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318b50) [ CSSD]2013-05-23 15:35:54.875 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318b50), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:54.875 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23182e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:35:54.875 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 630 (0x7f431bfd2b10) [ CSSD]2013-05-23 15:35:54.875 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2b10 (RPC#630) tag(276002c) sent to node 1 [ CSSD]2013-05-23 15:35:54.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0276002c) incarnation(174631051) [ CSSD]2013-05-23 15:35:54.875 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:35:54.875 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2b10 (RPC#630) state 6, flags 0x100 [ CSSD]2013-05-23 15:35:54.875 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2b10, ret 0, client 0x2318b50 member 0x23182e0 [ CSSD]2013-05-23 15:35:54.875 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 630 [ CSSD]2013-05-23 15:35:54.875 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23182e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:35:55.078 [1191196992] >TRACE: clssgmDeadProc: proc 0x23183f0 [ CSSD]2013-05-23 15:35:55.078 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23183f0) con(0x22ecae0) skgpid ospid 9855 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:35:55.965 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:56.187 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:56.187 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:57.694 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:35:57.694 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:35:57.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:35:58.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:35:58.211 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:35:59.981 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:00.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:00.222 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:02.054 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:02.227 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:02.227 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:02.697 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:02.697 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:03.557 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:36:03.557 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 631 (0x7f431bfd2b90) [ CSSD]2013-05-23 15:36:03.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2b90 (RPC#631) tag(277002c) sent to node 1 [ CSSD]2013-05-23 15:36:03.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:03.558 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 631 (0x7f431bfd2b90) [ CSSD]2013-05-23 15:36:03.558 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2b90 (RPC#631) state 6, flags 0x202 [ CSSD]2013-05-23 15:36:03.558 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2b90, ret 0, client (nil) [ CSSD]2013-05-23 15:36:03.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 631 [ CSSD]2013-05-23 15:36:03.827 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:03.827 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014012, gin 1, member 0 [ CSSD]2013-05-23 15:36:03.827 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:36:03.827 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:36:03.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:03.836 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:36:03.836 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:36:04.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:04.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:04.241 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:05.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:05.328 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014013, gin 1, member 0 [ CSSD]2013-05-23 15:36:05.329 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:36:05.329 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:36:05.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:05.335 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:36:05.335 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:36:06.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:06.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:06.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:07.559 [1191196992] >TRACE: clssgmAllocProc: (0x23183f0) allocated [ CSSD]2013-05-23 15:36:07.560 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:36:07.561 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23183f0 - 1,2,3 [ CSSD]2013-05-23 15:36:07.561 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22eca10) proc(0x23183f0) pid(9973) version 11:1:1:4 [ CSSD]2013-05-23 15:36:07.567 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x23183f0), client(1/0x2318aa0) [ CSSD]2013-05-23 15:36:07.567 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318aa0) [ CSSD]2013-05-23 15:36:07.567 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318aa0 with con 0x22ee130, requested num -1 [ CSSD]2013-05-23 15:36:07.567 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:07.567 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 632 (0x7f431bfd2c10) [ CSSD]2013-05-23 15:36:07.567 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2c10 (RPC#632) tag(278002c) sent to node 1 [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0278002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014014, gin 1, member 0 [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2c10 (RPC#632) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2c10, ret 0, client 0x2318aa0 member 0x23182e0 [ CSSD]2013-05-23 15:36:07.568 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318aa0 [ CSSD]2013-05-23 15:36:07.569 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 632 [ CSSD]2013-05-23 15:36:07.575 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318aa0) [ CSSD]2013-05-23 15:36:07.575 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318aa0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:07.575 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23182e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:07.575 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 633 (0x7f431bfd2c90) [ CSSD]2013-05-23 15:36:07.575 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2c90 (RPC#633) tag(279002c) sent to node 1 [ CSSD]2013-05-23 15:36:07.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0279002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:07.575 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:07.575 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2c90 (RPC#633) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:07.575 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2c90, ret 0, client 0x2318aa0 member 0x23182e0 [ CSSD]2013-05-23 15:36:07.576 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 633 [ CSSD]2013-05-23 15:36:07.576 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23182e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:36:07.700 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:07.700 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:07.778 [1191196992] >TRACE: clssgmDeadProc: proc 0x23183f0 [ CSSD]2013-05-23 15:36:07.778 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23183f0) con(0x22eca10) skgpid ospid 9973 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:08.097 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:08.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:08.252 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:10.103 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:10.110 [1191196992] >TRACE: clssgmAllocProc: (0x23183f0) allocated [ CSSD]2013-05-23 15:36:10.111 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:36:10.112 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23183f0 - 1,2,3 [ CSSD]2013-05-23 15:36:10.112 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecae0) proc(0x23183f0) pid(10003) version 11:1:1:4 [ CSSD]2013-05-23 15:36:10.118 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x23183f0), client(1/0x2318b50) [ CSSD]2013-05-23 15:36:10.118 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318b50) [ CSSD]2013-05-23 15:36:10.118 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318b50 with con 0x2318650, requested num -1 [ CSSD]2013-05-23 15:36:10.118 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:10.118 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 634 (0x7f431bfd2d10) [ CSSD]2013-05-23 15:36:10.118 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2d10 (RPC#634) tag(27a002c) sent to node 1 [ CSSD]2013-05-23 15:36:10.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(027a002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:10.118 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014015, gin 1, member 0 [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318cc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2d10 (RPC#634) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2d10, ret 0, client 0x2318b50 member 0x23182e0 [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318b50 [ CSSD]2013-05-23 15:36:10.119 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 634 [ CSSD]2013-05-23 15:36:10.125 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318b50) [ CSSD]2013-05-23 15:36:10.125 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318b50), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:10.125 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23182e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:10.125 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 635 (0x7f431bfd2d90) [ CSSD]2013-05-23 15:36:10.125 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2d90 (RPC#635) tag(27b002c) sent to node 1 [ CSSD]2013-05-23 15:36:10.125 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(027b002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:10.125 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:10.125 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2d90 (RPC#635) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:10.125 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2d90, ret 0, client 0x2318b50 member 0x23182e0 [ CSSD]2013-05-23 15:36:10.125 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 635 [ CSSD]2013-05-23 15:36:10.125 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23182e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:36:10.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:10.285 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:10.328 [1191196992] >TRACE: clssgmDeadProc: proc 0x23183f0 [ CSSD]2013-05-23 15:36:10.328 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23183f0) con(0x22ecae0) skgpid ospid 10003 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:12.155 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:12.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:12.291 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:12.704 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:12.704 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:13.581 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:36:13.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 636 (0x7f431bfd2e10) [ CSSD]2013-05-23 15:36:13.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2e10 (RPC#636) tag(27c002c) sent to node 1 [ CSSD]2013-05-23 15:36:13.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:13.582 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 636 (0x7f431bfd2e10) [ CSSD]2013-05-23 15:36:13.582 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2e10 (RPC#636) state 6, flags 0x202 [ CSSD]2013-05-23 15:36:13.582 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2e10, ret 0, client (nil) [ CSSD]2013-05-23 15:36:13.582 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 636 [ CSSD]2013-05-23 15:36:14.181 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:14.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:14.302 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:14.489 [1191196992] >TRACE: clssgmAllocProc: (0x23183f0) allocated [ CSSD]2013-05-23 15:36:14.490 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:36:14.491 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23183f0 - 1,2,3 [ CSSD]2013-05-23 15:36:14.491 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22eca10) proc(0x23183f0) pid(10089) version 11:1:1:4 [ CSSD]2013-05-23 15:36:14.508 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x23183f0), client(1/0x2318aa0) [ CSSD]2013-05-23 15:36:14.508 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318aa0) [ CSSD]2013-05-23 15:36:14.508 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x2318aa0 with con 0x22ee130, requested num -1 [ CSSD]2013-05-23 15:36:14.508 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:36:14.508 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 637 (0x7f431bfd2e90) [ CSSD]2013-05-23 15:36:14.508 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2e90 (RPC#637) tag(27d002c) sent to node 1 [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(027d002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014016, gin 1, member 0 [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318c10/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2e90 (RPC#637) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2e90, ret 0, client 0x2318aa0 member 0x23182e0 [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318aa0 [ CSSD]2013-05-23 15:36:14.509 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 637 [ CSSD]2013-05-23 15:36:14.515 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318aa0) [ CSSD]2013-05-23 15:36:14.515 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318aa0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:36:14.515 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23182e0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:36:14.515 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 638 (0x7f431bfd2f10) [ CSSD]2013-05-23 15:36:14.515 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2f10 (RPC#638) tag(27e002c) sent to node 1 [ CSSD]2013-05-23 15:36:14.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(027e002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:14.516 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:36:14.516 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2f10 (RPC#638) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:14.516 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2f10, ret 0, client 0x2318aa0 member 0x23182e0 [ CSSD]2013-05-23 15:36:14.516 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 638 [ CSSD]2013-05-23 15:36:14.516 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x23182e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:36:14.627 [1191196992] >TRACE: clssgmAllocProc: (0x22e4ad0) allocated [ CSSD]2013-05-23 15:36:14.628 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:36:14.629 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4ad0 - 1,2,3 [ CSSD]2013-05-23 15:36:14.629 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2318b10) proc(0x22e4ad0) pid(10094) version 11:1:1:4 [ CSSD]2013-05-23 15:36:14.643 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:36:14.645 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:36:14.647 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4ad0 [ CSSD]2013-05-23 15:36:14.647 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4ad0) con(0x2318b10) skgpid ospid 10094 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:14.718 [1191196992] >TRACE: clssgmDeadProc: proc 0x23183f0 [ CSSD]2013-05-23 15:36:14.718 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23183f0) con(0x22eca10) skgpid ospid 10089 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:16.187 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:16.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:16.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:17.707 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:17.707 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:18.196 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:18.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:18.313 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:20.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:20.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:20.335 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:20.588 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:20.588 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014017, gin 1, member 0 [ CSSD]2013-05-23 15:36:20.588 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:36:20.588 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:36:20.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:20.595 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:36:20.595 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:36:21.710 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:21.710 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:36:22.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:22.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:22.340 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:22.787 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x231d8a0) [ CSSD]2013-05-23 15:36:22.787 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x231d8a0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:36:22.787 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x231d8a0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:36:22.787 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 15:36:22.787 [1191196992] >TRACE: clssgmDeadProc: proc 0x2324250 [ CSSD]2013-05-23 15:36:22.787 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2324250) con(0x22f84f0) skgpid 4291 ospid 4291 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:23.605 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:36:23.606 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 639 (0x7f431bfd2f90) [ CSSD]2013-05-23 15:36:23.606 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2f90 (RPC#639) tag(27f002c) sent to node 1 [ CSSD]2013-05-23 15:36:23.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:23.606 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 639 (0x7f431bfd2f90) [ CSSD]2013-05-23 15:36:23.606 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2f90 (RPC#639) state 6, flags 0x202 [ CSSD]2013-05-23 15:36:23.606 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2f90, ret 0, client (nil) [ CSSD]2013-05-23 15:36:23.607 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 639 [ CSSD]2013-05-23 15:36:24.261 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:24.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:24.393 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:25.361 [1191196992] >TRACE: clssgmAllocProc: (0x22dfd50) allocated [ CSSD]2013-05-23 15:36:25.362 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:36:25.362 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dfd50 - 1,2,3 [ CSSD]2013-05-23 15:36:25.362 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f84e0) proc(0x22dfd50) pid(10190) version 11:1:1:4 [ CSSD]2013-05-23 15:36:25.379 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22dfd50), client(1/0x231d970) [ CSSD]2013-05-23 15:36:25.379 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231d970) [ CSSD]2013-05-23 15:36:25.379 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231d970 with con 0x22dff50, requested num -1 [ CSSD]2013-05-23 15:36:25.379 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:25.379 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 640 (0x7f431bfd3010) [ CSSD]2013-05-23 15:36:25.379 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3010 (RPC#640) tag(280002c) sent to node 1 [ CSSD]2013-05-23 15:36:25.379 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0280002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:25.379 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014018, gin 1, member 0 [ CSSD]2013-05-23 15:36:25.379 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec830/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:36:25.379 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:36:25.380 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:36:25.380 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3010 (RPC#640) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:25.380 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3010, ret 0, client 0x231d970 member 0x22f8670 [ CSSD]2013-05-23 15:36:25.380 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231d970 [ CSSD]2013-05-23 15:36:25.380 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 640 [ CSSD]2013-05-23 15:36:25.386 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231d970) [ CSSD]2013-05-23 15:36:25.386 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231d970), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:25.386 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f8670) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:25.386 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 641 (0x7f431bfd3090) [ CSSD]2013-05-23 15:36:25.386 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3090 (RPC#641) tag(281002c) sent to node 1 [ CSSD]2013-05-23 15:36:25.387 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0281002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:25.387 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:25.387 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3090 (RPC#641) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:25.387 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3090, ret 0, client 0x231d970 member 0x22f8670 [ CSSD]2013-05-23 15:36:25.387 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 641 [ CSSD]2013-05-23 15:36:25.387 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f8670) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:36:25.590 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dfd50 [ CSSD]2013-05-23 15:36:25.590 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dfd50) con(0x22f84e0) skgpid ospid 10190 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:26.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:26.399 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:26.399 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:26.713 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:26.713 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:28.298 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:28.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:28.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:30.304 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:30.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:30.415 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:31.716 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:31.717 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:32.355 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:32.420 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:32.420 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:33.630 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:36:33.630 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 642 (0x7f431bfd3110) [ CSSD]2013-05-23 15:36:33.630 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3110 (RPC#642) tag(282002c) sent to node 1 [ CSSD]2013-05-23 15:36:33.631 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:33.631 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 642 (0x7f431bfd3110) [ CSSD]2013-05-23 15:36:33.631 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3110 (RPC#642) state 6, flags 0x202 [ CSSD]2013-05-23 15:36:33.631 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3110, ret 0, client (nil) [ CSSD]2013-05-23 15:36:33.631 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 642 [ CSSD]2013-05-23 15:36:34.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:34.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:34.432 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:35.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:35.837 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014019, gin 1, member 0 [ CSSD]2013-05-23 15:36:35.837 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:36:35.837 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:36:35.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:35.844 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:36:35.844 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:36:36.367 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:36.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:36.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:36.720 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:36.720 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:38.379 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:38.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:38.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:40.383 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:40.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:40.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:40.623 [1191196992] >TRACE: clssgmAllocProc: (0x22dfd50) allocated [ CSSD]2013-05-23 15:36:40.624 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:36:40.625 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dfd50 - 1,2,3 [ CSSD]2013-05-23 15:36:40.625 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f84e0) proc(0x22dfd50) pid(10353) version 11:1:1:4 [ CSSD]2013-05-23 15:36:40.631 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22dfd50), client(1/0x231d970) [ CSSD]2013-05-23 15:36:40.631 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231d970) [ CSSD]2013-05-23 15:36:40.631 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231d970 with con 0x22dff50, requested num -1 [ CSSD]2013-05-23 15:36:40.631 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:40.631 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 643 (0x7f431bfd3190) [ CSSD]2013-05-23 15:36:40.631 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3190 (RPC#643) tag(283002c) sent to node 1 [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0283002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014020, gin 1, member 0 [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec830/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3190 (RPC#643) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3190, ret 0, client 0x231d970 member 0x22f8670 [ CSSD]2013-05-23 15:36:40.631 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231d970 [ CSSD]2013-05-23 15:36:40.632 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 643 [ CSSD]2013-05-23 15:36:40.638 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231d970) [ CSSD]2013-05-23 15:36:40.638 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231d970), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:40.638 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f8670) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:40.638 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 644 (0x7f431bfd3210) [ CSSD]2013-05-23 15:36:40.638 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3210 (RPC#644) tag(284002c) sent to node 1 [ CSSD]2013-05-23 15:36:40.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0284002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:40.638 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:40.638 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3210 (RPC#644) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:40.638 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3210, ret 0, client 0x231d970 member 0x22f8670 [ CSSD]2013-05-23 15:36:40.638 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 644 [ CSSD]2013-05-23 15:36:40.638 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f8670) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:36:40.840 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dfd50 [ CSSD]2013-05-23 15:36:40.840 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dfd50) con(0x22f84e0) skgpid ospid 10353 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:41.723 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:41.723 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:42.396 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:42.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:42.520 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:43.652 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:36:43.652 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 645 (0x7f431bfd3290) [ CSSD]2013-05-23 15:36:43.652 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3290 (RPC#645) tag(285002c) sent to node 1 [ CSSD]2013-05-23 15:36:43.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:43.653 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 645 (0x7f431bfd3290) [ CSSD]2013-05-23 15:36:43.653 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3290 (RPC#645) state 6, flags 0x202 [ CSSD]2013-05-23 15:36:43.653 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3290, ret 0, client (nil) [ CSSD]2013-05-23 15:36:43.653 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 645 [ CSSD]2013-05-23 15:36:44.402 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:44.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:44.533 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:46.427 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:46.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:46.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:46.726 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:46.726 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:48.477 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:48.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:48.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:50.482 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:50.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:50.598 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:51.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:51.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014021, gin 1, member 0 [ CSSD]2013-05-23 15:36:51.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:36:51.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:36:51.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:51.111 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:36:51.111 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:36:51.729 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:51.730 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:52.496 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:52.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:52.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:53.683 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:36:53.683 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 646 (0x7f431bfd3310) [ CSSD]2013-05-23 15:36:53.683 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3310 (RPC#646) tag(286002c) sent to node 1 [ CSSD]2013-05-23 15:36:53.684 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:53.684 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 646 (0x7f431bfd3310) [ CSSD]2013-05-23 15:36:53.684 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3310 (RPC#646) state 6, flags 0x202 [ CSSD]2013-05-23 15:36:53.684 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3310, ret 0, client (nil) [ CSSD]2013-05-23 15:36:53.684 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 646 [ CSSD]2013-05-23 15:36:54.502 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:54.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:54.613 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:55.882 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 15:36:55.883 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:36:55.884 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 15:36:55.884 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(10519) version 11:1:1:4 [ CSSD]2013-05-23 15:36:55.889 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 15:36:55.889 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:36:55.889 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c109f70, requested num -1 [ CSSD]2013-05-23 15:36:55.890 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:55.890 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 647 (0x7f431bfd3390) [ CSSD]2013-05-23 15:36:55.890 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3390 (RPC#647) tag(287002c) sent to node 1 [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0287002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014022, gin 1, member 0 [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3390 (RPC#647) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3390, ret 0, client 0x7f431c0e6e30 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:36:55.890 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 15:36:55.891 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 647 [ CSSD]2013-05-23 15:36:55.897 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:36:55.897 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:55.897 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1247a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:36:55.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 648 (0x7f431bfd3410) [ CSSD]2013-05-23 15:36:55.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3410 (RPC#648) tag(288002c) sent to node 1 [ CSSD]2013-05-23 15:36:55.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0288002c) incarnation(174631051) [ CSSD]2013-05-23 15:36:55.898 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:36:55.898 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3410 (RPC#648) state 6, flags 0x100 [ CSSD]2013-05-23 15:36:55.898 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3410, ret 0, client 0x7f431c0e6e30 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:36:55.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 648 [ CSSD]2013-05-23 15:36:55.898 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1247a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:36:56.100 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 15:36:56.100 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 10519 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:36:56.508 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:56.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:56.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:36:56.733 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:36:56.733 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:36:58.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:36:58.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:36:58.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:00.534 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:00.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:00.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:01.735 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:01.736 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:02.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:02.662 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:02.662 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:03.708 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:37:03.708 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 649 (0x7f431bfd3490) [ CSSD]2013-05-23 15:37:03.709 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3490 (RPC#649) tag(289002c) sent to node 1 [ CSSD]2013-05-23 15:37:03.709 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:03.709 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 649 (0x7f431bfd3490) [ CSSD]2013-05-23 15:37:03.709 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3490 (RPC#649) state 6, flags 0x202 [ CSSD]2013-05-23 15:37:03.709 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3490, ret 0, client (nil) [ CSSD]2013-05-23 15:37:03.709 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 649 [ CSSD]2013-05-23 15:37:04.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:04.094 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014023, gin 1, member 0 [ CSSD]2013-05-23 15:37:04.094 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:04.095 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:04.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:04.102 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:04.102 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:04.571 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:04.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:04.714 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:06.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:06.376 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014024, gin 1, member 0 [ CSSD]2013-05-23 15:37:06.376 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:06.376 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:06.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:06.384 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:06.384 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:06.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:06.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:06.719 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:06.738 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:06.738 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:07.823 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 15:37:07.824 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:37:07.824 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 15:37:07.824 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c108370) pid(10642) version 11:1:1:4 [ CSSD]2013-05-23 15:37:07.830 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:37:07.830 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:37:07.830 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:37:07.830 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:07.830 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 650 (0x7f431bfd3510) [ CSSD]2013-05-23 15:37:07.830 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3510 (RPC#650) tag(28a002c) sent to node 1 [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(028a002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014025, gin 1, member 0 [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3510 (RPC#650) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3510, ret 0, client 0x7f431c135120 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:37:07.831 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 650 [ CSSD]2013-05-23 15:37:07.838 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:37:07.838 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:07.838 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1247a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:07.838 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 651 (0x7f431bfd3590) [ CSSD]2013-05-23 15:37:07.838 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3590 (RPC#651) tag(28b002c) sent to node 1 [ CSSD]2013-05-23 15:37:07.838 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(028b002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:07.838 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:07.838 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3590 (RPC#651) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:07.838 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3590, ret 0, client 0x7f431c135120 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:37:07.838 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 651 [ CSSD]2013-05-23 15:37:07.838 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1247a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:37:08.041 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 15:37:08.041 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c109f70) skgpid ospid 10642 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:08.607 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:08.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:08.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:10.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:10.519 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014026, gin 1, member 0 [ CSSD]2013-05-23 15:37:10.519 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:10.519 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:10.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:10.526 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:10.526 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:10.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:10.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:10.738 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:11.132 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 15:37:11.133 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:37:11.134 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 15:37:11.134 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(10677) version 11:1:1:4 [ CSSD]2013-05-23 15:37:11.150 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 15:37:11.150 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:37:11.150 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c109f70, requested num -1 [ CSSD]2013-05-23 15:37:11.150 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:11.150 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 652 (0x7f431bfd3610) [ CSSD]2013-05-23 15:37:11.150 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3610 (RPC#652) tag(28c002c) sent to node 1 [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(028c002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014027, gin 1, member 0 [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3610 (RPC#652) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3610, ret 0, client 0x7f431c0e6e30 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 15:37:11.151 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 652 [ CSSD]2013-05-23 15:37:11.158 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:37:11.158 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:11.158 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1247a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:11.158 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 653 (0x7f431bfd3690) [ CSSD]2013-05-23 15:37:11.158 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3690 (RPC#653) tag(28d002c) sent to node 1 [ CSSD]2013-05-23 15:37:11.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(028d002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:11.159 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:11.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3690 (RPC#653) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:11.159 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3690, ret 0, client 0x7f431c0e6e30 member 0x7f431c1247a0 [ CSSD]2013-05-23 15:37:11.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 653 [ CSSD]2013-05-23 15:37:11.159 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1247a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:37:11.361 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 15:37:11.361 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 10677 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:11.464 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 15:37:11.465 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:37:11.466 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 15:37:11.466 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108370) pid(9479/9479) version 11:1:1:4 [ CSSD]2013-05-23 15:37:11.482 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:37:11.482 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:37:11.482 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:37:11.482 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 15:37:11.741 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:11.741 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:11.839 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 15:37:11.840 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:37:11.841 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 15:37:11.841 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c12b920) pid(10167/10167) version 11:1:1:4 [ CSSD]2013-05-23 15:37:11.846 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 15:37:11.846 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:37:11.846 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:37:11.846 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 15:37:12.394 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(2/0x7f431c10a560) [ CSSD]2013-05-23 15:37:12.394 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c10a560) [ CSSD]2013-05-23 15:37:12.394 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:37:12.394 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 15:37:12.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:12.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:12.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:12.754 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129a90) allocated [ CSSD]2013-05-23 15:37:12.755 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:37:12.756 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129a90 - 1,2,3 [ CSSD]2013-05-23 15:37:12.756 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108b00) proc(0x7f431c129a90) pid(10788) version 11:1:1:4 [ CSSD]2013-05-23 15:37:12.882 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1343c0) allocated [ CSSD]2013-05-23 15:37:12.883 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:37:12.884 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1343c0 - 1,2,3 [ CSSD]2013-05-23 15:37:12.884 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c121130) proc(0x7f431c1343c0) pid(10798) version 11:1:1:4 [ CSSD]2013-05-23 15:37:12.899 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:37:12.901 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:37:12.903 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1343c0 [ CSSD]2013-05-23 15:37:12.903 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1343c0) con(0x7f431c121130) skgpid ospid 10798 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:12.976 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(58/0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:12.976 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 58 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:12.976 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c12e6d0 with con 0x7f431c1345e0, requested num -1 [ CSSD]2013-05-23 15:37:12.976 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:37:12.976 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 654 (0x7f431bfd3710) [ CSSD]2013-05-23 15:37:12.976 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3710 (RPC#654) tag(28e002c) sent to node 1 [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(028e002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014028, gin 1, member 0 [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c120ba0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3710 (RPC#654) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3710, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:37:12.977 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 654 [ CSSD]2013-05-23 15:37:12.984 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 58 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:12.984 [1191196992] >TRACE: clssgmExitGrock: client 58 (0x7f431c12e6d0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:37:12.984 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129890) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:37:12.984 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 655 (0x7f431bfd3790) [ CSSD]2013-05-23 15:37:12.984 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3790 (RPC#655) tag(28f002c) sent to node 1 [ CSSD]2013-05-23 15:37:12.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(028f002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:12.984 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:37:12.984 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3790 (RPC#655) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:12.984 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3790, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:12.984 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 655 [ CSSD]2013-05-23 15:37:12.984 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c129890) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:37:12.985 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129a90 [ CSSD]2013-05-23 15:37:12.985 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129a90) con(0x7f431c108b00) skgpid ospid 10788 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:13.730 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:37:13.730 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 656 (0x7f431bfd3810) [ CSSD]2013-05-23 15:37:13.730 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3810 (RPC#656) tag(290002c) sent to node 1 [ CSSD]2013-05-23 15:37:13.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:13.731 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 656 (0x7f431bfd3810) [ CSSD]2013-05-23 15:37:13.731 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3810 (RPC#656) state 6, flags 0x202 [ CSSD]2013-05-23 15:37:13.731 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3810, ret 0, client (nil) [ CSSD]2013-05-23 15:37:13.731 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 656 [ CSSD]2013-05-23 15:37:14.672 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:14.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:14.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:16.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:16.744 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:16.744 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:16.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:16.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:18.708 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:18.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:18.813 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:20.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:20.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:20.819 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:21.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:21.629 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014029, gin 1, member 0 [ CSSD]2013-05-23 15:37:21.629 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1348f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:21.629 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:21.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:21.636 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:21.636 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108830) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:21.748 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:21.748 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:22.768 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:22.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:22.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:23.756 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:37:23.756 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 657 (0x7f431bfd3890) [ CSSD]2013-05-23 15:37:23.756 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3890 (RPC#657) tag(291002c) sent to node 1 [ CSSD]2013-05-23 15:37:23.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:23.757 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 657 (0x7f431bfd3890) [ CSSD]2013-05-23 15:37:23.757 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3890 (RPC#657) state 6, flags 0x202 [ CSSD]2013-05-23 15:37:23.757 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3890, ret 0, client (nil) [ CSSD]2013-05-23 15:37:23.757 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 657 [ CSSD]2013-05-23 15:37:24.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:24.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:24.834 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:25.750 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:25.751 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:37:26.397 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129a90) allocated [ CSSD]2013-05-23 15:37:26.398 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:37:26.399 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129a90 - 1,2,3 [ CSSD]2013-05-23 15:37:26.399 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108910) proc(0x7f431c129a90) pid(11040) version 11:1:1:4 [ CSSD]2013-05-23 15:37:26.405 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129a90), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:26.405 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:26.405 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c120bc0, requested num -1 [ CSSD]2013-05-23 15:37:26.405 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:26.405 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 658 (0x7f431bfd3910) [ CSSD]2013-05-23 15:37:26.405 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3910 (RPC#658) tag(292002c) sent to node 1 [ CSSD]2013-05-23 15:37:26.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0292002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:26.405 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014030, gin 1, member 0 [ CSSD]2013-05-23 15:37:26.405 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c121190/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:37:26.406 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:37:26.406 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:37:26.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3910 (RPC#658) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:26.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3910, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:26.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:37:26.406 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 658 [ CSSD]2013-05-23 15:37:26.412 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:26.412 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:26.412 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129890) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:26.412 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 659 (0x7f431bfd3990) [ CSSD]2013-05-23 15:37:26.412 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3990 (RPC#659) tag(293002c) sent to node 1 [ CSSD]2013-05-23 15:37:26.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0293002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:26.413 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:26.413 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3990 (RPC#659) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:26.413 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3990, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:26.413 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 659 [ CSSD]2013-05-23 15:37:26.413 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129890) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:37:26.616 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129a90 [ CSSD]2013-05-23 15:37:26.616 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129a90) con(0x7f431c108910) skgpid ospid 11040 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:26.780 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:26.839 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:26.839 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:28.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:28.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:28.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:30.754 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:30.754 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:30.815 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:30.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:30.899 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:32.868 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:32.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:32.904 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:33.782 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:37:33.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 660 (0x7f431bfd3a10) [ CSSD]2013-05-23 15:37:33.782 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3a10 (RPC#660) tag(294002c) sent to node 1 [ CSSD]2013-05-23 15:37:33.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:33.783 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 660 (0x7f431bfd3a10) [ CSSD]2013-05-23 15:37:33.783 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3a10 (RPC#660) state 6, flags 0x202 [ CSSD]2013-05-23 15:37:33.783 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3a10, ret 0, client (nil) [ CSSD]2013-05-23 15:37:33.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 660 [ CSSD]2013-05-23 15:37:34.874 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:34.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:34.915 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:35.757 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:35.758 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:36.879 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:36.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:36.889 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014031, gin 1, member 0 [ CSSD]2013-05-23 15:37:36.890 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1348f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:36.890 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:36.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:36.896 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:36.896 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108920) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:36.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:36.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:38.890 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:38.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:38.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:40.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:40.039 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014032, gin 1, member 0 [ CSSD]2013-05-23 15:37:40.039 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1348f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:40.039 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:40.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:40.046 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:40.046 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108920) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:40.760 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:40.761 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:40.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:40.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:40.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:41.648 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129a90) allocated [ CSSD]2013-05-23 15:37:41.648 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:37:41.649 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129a90 - 1,2,3 [ CSSD]2013-05-23 15:37:41.649 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108910) proc(0x7f431c129a90) pid(11181) version 11:1:1:4 [ CSSD]2013-05-23 15:37:41.655 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129a90), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:41.655 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:41.655 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c120bc0, requested num -1 [ CSSD]2013-05-23 15:37:41.655 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:41.655 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 661 (0x7f431bfd3a90) [ CSSD]2013-05-23 15:37:41.655 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3a90 (RPC#661) tag(295002c) sent to node 1 [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0295002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014033, gin 1, member 0 [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c121190/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3a90 (RPC#661) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3a90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:37:41.656 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 661 [ CSSD]2013-05-23 15:37:41.663 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:41.663 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:41.663 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129890) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:41.663 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 662 (0x7f431bfd3b10) [ CSSD]2013-05-23 15:37:41.663 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3b10 (RPC#662) tag(296002c) sent to node 1 [ CSSD]2013-05-23 15:37:41.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0296002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:41.663 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:41.663 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3b10 (RPC#662) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:41.663 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3b10, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:41.663 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 662 [ CSSD]2013-05-23 15:37:41.663 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129890) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:37:41.866 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129a90 [ CSSD]2013-05-23 15:37:41.866 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129a90) con(0x7f431c108910) skgpid ospid 11181 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:42.954 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:42.954 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:42.968 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:43.811 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:37:43.811 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 663 (0x7f431bfd3b90) [ CSSD]2013-05-23 15:37:43.812 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3b90 (RPC#663) tag(297002c) sent to node 1 [ CSSD]2013-05-23 15:37:43.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:43.812 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 663 (0x7f431bfd3b90) [ CSSD]2013-05-23 15:37:43.812 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3b90 (RPC#663) state 6, flags 0x202 [ CSSD]2013-05-23 15:37:43.812 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3b90, ret 0, client (nil) [ CSSD]2013-05-23 15:37:43.812 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 663 [ CSSD]2013-05-23 15:37:44.763 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:44.763 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:37:44.974 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:45.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:45.006 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:46.986 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:47.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:47.050 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:48.499 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 15:37:48.499 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2312950), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:37:48.499 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2312950), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:37:48.499 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 15:37:48.499 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:37:48.499 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x2302f30) skgpid 5732 ospid 5732 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:48.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:49.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:49.104 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:49.766 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:49.767 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:51.036 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:51.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:51.110 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:52.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:52.139 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014034, gin 1, member 0 [ CSSD]2013-05-23 15:37:52.140 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1348f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:37:52.140 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:37:52.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:52.146 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:37:52.146 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108920) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:37:53.087 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:53.115 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:53.116 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:53.841 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:37:53.841 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 664 (0x7f431bfd3c10) [ CSSD]2013-05-23 15:37:53.841 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3c10 (RPC#664) tag(298002c) sent to node 1 [ CSSD]2013-05-23 15:37:53.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:53.842 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 664 (0x7f431bfd3c10) [ CSSD]2013-05-23 15:37:53.842 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3c10 (RPC#664) state 6, flags 0x202 [ CSSD]2013-05-23 15:37:53.842 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3c10, ret 0, client (nil) [ CSSD]2013-05-23 15:37:53.842 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 664 [ CSSD]2013-05-23 15:37:54.769 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:54.769 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:37:55.092 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:55.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:55.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:56.898 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129a90) allocated [ CSSD]2013-05-23 15:37:56.899 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:37:56.899 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129a90 - 1,2,3 [ CSSD]2013-05-23 15:37:56.899 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108910) proc(0x7f431c129a90) pid(11431) version 11:1:1:4 [ CSSD]2013-05-23 15:37:56.905 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c129a90), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:56.905 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:56.905 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c120bc0, requested num -1 [ CSSD]2013-05-23 15:37:56.905 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:56.905 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 665 (0x7f431bfd3c90) [ CSSD]2013-05-23 15:37:56.906 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3c90 (RPC#665) tag(299002c) sent to node 1 [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0299002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014035, gin 1, member 0 [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c121190/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3c90 (RPC#665) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3c90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:56.906 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:37:56.907 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 665 [ CSSD]2013-05-23 15:37:56.913 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:37:56.913 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:56.913 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129890) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:37:56.913 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 666 (0x7f431bfd3d10) [ CSSD]2013-05-23 15:37:56.913 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3d10 (RPC#666) tag(29a002c) sent to node 1 [ CSSD]2013-05-23 15:37:56.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(029a002c) incarnation(174631051) [ CSSD]2013-05-23 15:37:56.913 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:37:56.913 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3d10 (RPC#666) state 6, flags 0x100 [ CSSD]2013-05-23 15:37:56.913 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3d10, ret 0, client 0x7f431c12e6d0 member 0x7f431c129890 [ CSSD]2013-05-23 15:37:56.914 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 666 [ CSSD]2013-05-23 15:37:56.914 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129890) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:37:57.107 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:57.116 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129a90 [ CSSD]2013-05-23 15:37:57.116 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129a90) con(0x7f431c108910) skgpid ospid 11431 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:37:57.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:57.133 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:59.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:37:59.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:37:59.205 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:37:59.772 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:37:59.772 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:01.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:01.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:01.230 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:03.169 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:03.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:03.236 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:03.872 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:38:03.872 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 667 (0x7f431bfd3d90) [ CSSD]2013-05-23 15:38:03.873 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3d90 (RPC#667) tag(29b002c) sent to node 1 [ CSSD]2013-05-23 15:38:03.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:03.873 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 667 (0x7f431bfd3d90) [ CSSD]2013-05-23 15:38:03.873 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3d90 (RPC#667) state 6, flags 0x202 [ CSSD]2013-05-23 15:38:03.873 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3d90, ret 0, client (nil) [ CSSD]2013-05-23 15:38:03.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 667 [ CSSD]2013-05-23 15:38:04.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:04.359 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014036, gin 1, member 0 [ CSSD]2013-05-23 15:38:04.359 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1348f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:38:04.359 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:38:04.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:04.366 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:38:04.367 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108830) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:38:04.775 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:04.776 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:05.194 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:05.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:05.246 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:06.435 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129a90) allocated [ CSSD]2013-05-23 15:38:06.436 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:38:06.436 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129a90 - 1,2,3 [ CSSD]2013-05-23 15:38:06.436 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108910) proc(0x7f431c129a90) pid(11345/11345) version 11:1:1:4 [ CSSD]2013-05-23 15:38:06.442 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c129a90), client(1/0x2302f30) [ CSSD]2013-05-23 15:38:06.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2302f30) [ CSSD]2013-05-23 15:38:06.442 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:38:06.442 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 15:38:06.496 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c129a90), client(2/0x2312950) [ CSSD]2013-05-23 15:38:06.496 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2312950) [ CSSD]2013-05-23 15:38:06.496 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:38:06.496 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 15:38:06.529 [1191196992] >TRACE: clssgmAllocProc: (0x22e2eb0) allocated [ CSSD]2013-05-23 15:38:06.530 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:38:06.530 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e2eb0 - 1,2,3 [ CSSD]2013-05-23 15:38:06.530 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22fdb60) proc(0x22e2eb0) pid(11343/11343) version 11:1:1:4 [ CSSD]2013-05-23 15:38:06.535 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e2eb0), client(1/0x22e2fe0) [ CSSD]2013-05-23 15:38:06.535 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e2fe0) [ CSSD]2013-05-23 15:38:06.535 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:38:06.535 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 15:38:06.556 [1191196992] >TRACE: clssgmAllocProc: (0x23024c0) allocated [ CSSD]2013-05-23 15:38:06.556 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:38:06.556 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23024c0 - 1,2,3 [ CSSD]2013-05-23 15:38:06.556 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f84f0) proc(0x23024c0) pid(10165/10165) version 11:1:1:4 [ CSSD]2013-05-23 15:38:06.561 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x23024c0), client(1/0x23029f0) [ CSSD]2013-05-23 15:38:06.561 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23029f0) [ CSSD]2013-05-23 15:38:06.561 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:38:06.561 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 15:38:06.814 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e2eb0), client(2/0x22ed3f0) [ CSSD]2013-05-23 15:38:06.814 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22ed3f0) [ CSSD]2013-05-23 15:38:06.814 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:38:06.814 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 28 [ CSSD]2013-05-23 15:38:07.021 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x23024c0), client(2/0x22ed830) [ CSSD]2013-05-23 15:38:07.021 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22ed830) [ CSSD]2013-05-23 15:38:07.021 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:38:07.021 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 15:38:07.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:07.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:07.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:07.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:07.388 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014037, gin 1, member 0 [ CSSD]2013-05-23 15:38:07.388 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:38:07.389 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:38:07.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:07.395 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:38:07.395 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:38:08.078 [1191196992] >TRACE: clssgmAllocProc: (0x22ed6f0) allocated [ CSSD]2013-05-23 15:38:08.079 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:38:08.079 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed6f0 - 1,2,3 [ CSSD]2013-05-23 15:38:08.079 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee030) proc(0x22ed6f0) pid(11545) version 11:1:1:4 [ CSSD]2013-05-23 15:38:08.086 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed6f0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:38:08.086 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:08.086 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x2319180, requested num -1 [ CSSD]2013-05-23 15:38:08.086 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:08.086 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 668 (0x7f431bfd3e10) [ CSSD]2013-05-23 15:38:08.086 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3e10 (RPC#668) tag(29c002c) sent to node 1 [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(029c002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014038, gin 1, member 0 [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3e60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3e10 (RPC#668) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3e10, ret 0, client 0x22ee1b0 member 0x2318f30 [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:38:08.087 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 668 [ CSSD]2013-05-23 15:38:08.093 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:08.093 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:08.093 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318f30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:08.093 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 669 (0x7f431bfd3e90) [ CSSD]2013-05-23 15:38:08.093 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3e90 (RPC#669) tag(29d002c) sent to node 1 [ CSSD]2013-05-23 15:38:08.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(029d002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:08.094 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:08.094 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3e90 (RPC#669) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:08.094 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3e90, ret 0, client 0x22ee1b0 member 0x2318f30 [ CSSD]2013-05-23 15:38:08.094 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 669 [ CSSD]2013-05-23 15:38:08.094 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318f30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:38:08.296 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed6f0 [ CSSD]2013-05-23 15:38:08.296 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed6f0) con(0x22ee030) skgpid ospid 11545 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:38:08.778 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:08.778 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:38:09.213 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:09.305 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:09.305 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:11.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:11.310 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:11.310 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:12.148 [1191196992] >TRACE: clssgmAllocProc: (0x22ed6f0) allocated [ CSSD]2013-05-23 15:38:12.149 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:38:12.149 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed6f0 - 1,2,3 [ CSSD]2013-05-23 15:38:12.149 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee030) proc(0x22ed6f0) pid(11614) version 11:1:1:4 [ CSSD]2013-05-23 15:38:12.155 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed6f0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:38:12.155 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:12.155 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x2319180, requested num -1 [ CSSD]2013-05-23 15:38:12.155 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:12.155 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 670 (0x7f431bfd3f10) [ CSSD]2013-05-23 15:38:12.155 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3f10 (RPC#670) tag(29e002c) sent to node 1 [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(029e002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014039, gin 1, member 0 [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3e60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3f10 (RPC#670) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3f10, ret 0, client 0x22ee1b0 member 0x2318f30 [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:38:12.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 670 [ CSSD]2013-05-23 15:38:12.162 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:12.162 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:12.162 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318f30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:12.162 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 671 (0x7f431bfd3f90) [ CSSD]2013-05-23 15:38:12.162 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3f90 (RPC#671) tag(29f002c) sent to node 1 [ CSSD]2013-05-23 15:38:12.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(029f002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:12.162 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:12.162 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3f90 (RPC#671) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:12.162 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3f90, ret 0, client 0x22ee1b0 member 0x2318f30 [ CSSD]2013-05-23 15:38:12.163 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 671 [ CSSD]2013-05-23 15:38:12.163 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318f30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:38:12.365 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed6f0 [ CSSD]2013-05-23 15:38:12.365 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed6f0) con(0x22ee030) skgpid ospid 11614 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:38:13.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:13.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:13.403 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:13.781 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:13.782 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:13.897 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:38:13.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 672 (0x7f431bfd4010) [ CSSD]2013-05-23 15:38:13.898 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4010 (RPC#672) tag(2a0002c) sent to node 1 [ CSSD]2013-05-23 15:38:13.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:13.898 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 672 (0x7f431bfd4010) [ CSSD]2013-05-23 15:38:13.898 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4010 (RPC#672) state 6, flags 0x202 [ CSSD]2013-05-23 15:38:13.898 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4010, ret 0, client (nil) [ CSSD]2013-05-23 15:38:13.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 672 [ CSSD]2013-05-23 15:38:15.265 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:15.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:15.428 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:17.278 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:17.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:17.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:18.784 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:18.785 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:19.283 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:19.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:19.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:21.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:21.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:21.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:22.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:22.648 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014040, gin 1, member 0 [ CSSD]2013-05-23 15:38:22.648 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:38:22.648 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:38:22.655 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:22.655 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:38:22.655 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:38:23.310 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:23.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:23.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:23.788 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:23.788 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:23.920 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:38:23.920 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 673 (0x7f431bfd4090) [ CSSD]2013-05-23 15:38:23.921 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4090 (RPC#673) tag(2a1002c) sent to node 1 [ CSSD]2013-05-23 15:38:23.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:23.921 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 673 (0x7f431bfd4090) [ CSSD]2013-05-23 15:38:23.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4090 (RPC#673) state 6, flags 0x202 [ CSSD]2013-05-23 15:38:23.921 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4090, ret 0, client (nil) [ CSSD]2013-05-23 15:38:23.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 673 [ CSSD]2013-05-23 15:38:25.315 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:25.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:25.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:27.327 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:27.397 [1191196992] >TRACE: clssgmAllocProc: (0x2319050) allocated [ CSSD]2013-05-23 15:38:27.398 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:38:27.399 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2319050 - 1,2,3 [ CSSD]2013-05-23 15:38:27.399 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee030) proc(0x2319050) pid(11760) version 11:1:1:4 [ CSSD]2013-05-23 15:38:27.405 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2319050), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:38:27.405 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:27.405 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x23192b0, requested num -1 [ CSSD]2013-05-23 15:38:27.405 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:27.405 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 674 (0x7f431bfd4110) [ CSSD]2013-05-23 15:38:27.405 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4110 (RPC#674) tag(2a2002c) sent to node 1 [ CSSD]2013-05-23 15:38:27.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02a2002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:27.405 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014041, gin 1, member 0 [ CSSD]2013-05-23 15:38:27.405 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:38:27.405 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:38:27.406 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:38:27.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4110 (RPC#674) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:27.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4110, ret 0, client 0x22ee1b0 member 0x2318f10 [ CSSD]2013-05-23 15:38:27.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:38:27.406 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 674 [ CSSD]2013-05-23 15:38:27.411 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:27.411 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:27.411 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318f10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:27.411 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 675 (0x7f431bfd4190) [ CSSD]2013-05-23 15:38:27.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4190 (RPC#675) tag(2a3002c) sent to node 1 [ CSSD]2013-05-23 15:38:27.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02a3002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:27.412 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:27.412 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4190 (RPC#675) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:27.412 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4190, ret 0, client 0x22ee1b0 member 0x2318f10 [ CSSD]2013-05-23 15:38:27.412 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 675 [ CSSD]2013-05-23 15:38:27.412 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318f10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:38:27.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:27.514 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:27.614 [1191196992] >TRACE: clssgmDeadProc: proc 0x2319050 [ CSSD]2013-05-23 15:38:27.614 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2319050) con(0x22ee030) skgpid ospid 11760 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:38:27.790 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:27.790 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:38:29.333 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:29.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:29.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:31.358 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:31.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:31.551 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:32.794 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:32.794 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:33.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:33.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:33.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:33.947 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:38:33.947 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 676 (0x7f431bfd4210) [ CSSD]2013-05-23 15:38:33.947 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4210 (RPC#676) tag(2a4002c) sent to node 1 [ CSSD]2013-05-23 15:38:33.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:33.948 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 676 (0x7f431bfd4210) [ CSSD]2013-05-23 15:38:33.948 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4210 (RPC#676) state 6, flags 0x202 [ CSSD]2013-05-23 15:38:33.948 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4210, ret 0, client (nil) [ CSSD]2013-05-23 15:38:33.948 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 676 [ CSSD]2013-05-23 15:38:35.416 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:35.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:35.609 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:35.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:35.669 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014042, gin 1, member 0 [ CSSD]2013-05-23 15:38:35.669 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 15:38:35.669 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:38:35.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:35.675 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 15:38:35.675 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:38:35.800 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:38:35.801 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:38:35.803 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:38:35.803 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:38:35.804 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:38:35.805 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:38:37.428 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:37.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:37.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:37.797 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:37.797 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:37.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:37.909 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014043, gin 1, member 0 [ CSSD]2013-05-23 15:38:37.909 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:38:37.909 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:38:37.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:37.916 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:38:37.916 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:38:39.434 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:39.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:39.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:41.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:41.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:41.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:42.647 [1191196992] >TRACE: clssgmAllocProc: (0x22ed6f0) allocated [ CSSD]2013-05-23 15:38:42.648 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:38:42.649 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed6f0 - 1,2,3 [ CSSD]2013-05-23 15:38:42.649 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee030) proc(0x22ed6f0) pid(11919) version 11:1:1:4 [ CSSD]2013-05-23 15:38:42.655 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed6f0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:38:42.655 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:42.655 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x2319180, requested num -1 [ CSSD]2013-05-23 15:38:42.655 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:42.655 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 677 (0x7f431bfd4290) [ CSSD]2013-05-23 15:38:42.655 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4290 (RPC#677) tag(2a5002c) sent to node 1 [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02a5002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014044, gin 1, member 0 [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3e60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4290 (RPC#677) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4290, ret 0, client 0x22ee1b0 member 0x2318f30 [ CSSD]2013-05-23 15:38:42.655 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:38:42.656 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 677 [ CSSD]2013-05-23 15:38:42.663 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:38:42.663 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:42.663 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318f30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:42.663 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 678 (0x7f431bfd4310) [ CSSD]2013-05-23 15:38:42.663 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4310 (RPC#678) tag(2a6002c) sent to node 1 [ CSSD]2013-05-23 15:38:42.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02a6002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:42.663 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:42.663 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4310 (RPC#678) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:42.663 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4310, ret 0, client 0x22ee1b0 member 0x2318f30 [ CSSD]2013-05-23 15:38:42.664 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 678 [ CSSD]2013-05-23 15:38:42.664 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318f30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:38:42.800 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:42.800 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:42.866 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed6f0 [ CSSD]2013-05-23 15:38:42.866 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed6f0) con(0x22ee030) skgpid ospid 11919 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:38:43.491 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:43.644 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:43.644 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:43.970 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:38:43.970 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 679 (0x7f431bfd4390) [ CSSD]2013-05-23 15:38:43.970 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4390 (RPC#679) tag(2a7002c) sent to node 1 [ CSSD]2013-05-23 15:38:43.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:43.971 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 679 (0x7f431bfd4390) [ CSSD]2013-05-23 15:38:43.971 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4390 (RPC#679) state 6, flags 0x202 [ CSSD]2013-05-23 15:38:43.971 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4390, ret 0, client (nil) [ CSSD]2013-05-23 15:38:43.971 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 679 [ CSSD]2013-05-23 15:38:45.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:45.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:45.649 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:46.803 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:46.803 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:38:47.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:47.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:47.674 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:49.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:49.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:49.677 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:51.541 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:51.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:51.682 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:51.806 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:51.806 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:52.805 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f3fe0) [ CSSD]2013-05-23 15:38:52.805 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f3fe0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:38:52.805 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f3fe0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:38:52.805 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 15:38:52.815 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22eefa0) [ CSSD]2013-05-23 15:38:52.815 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22eefa0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:38:52.815 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22eefa0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:38:52.815 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 15:38:52.815 [1191196992] >TRACE: clssgmDeadProc: proc 0x22da8d0 [ CSSD]2013-05-23 15:38:52.815 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22da8d0) con(0x231da10) skgpid 6503 ospid 6503 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:38:53.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:53.158 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014045, gin 1, member 0 [ CSSD]2013-05-23 15:38:53.158 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:38:53.158 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:38:53.165 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:53.165 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:38:53.165 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:38:53.592 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:53.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:53.714 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:53.996 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:38:53.996 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 680 (0x7f431bfd4410) [ CSSD]2013-05-23 15:38:53.996 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4410 (RPC#680) tag(2a8002c) sent to node 1 [ CSSD]2013-05-23 15:38:53.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:53.997 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 680 (0x7f431bfd4410) [ CSSD]2013-05-23 15:38:53.997 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4410 (RPC#680) state 6, flags 0x202 [ CSSD]2013-05-23 15:38:53.997 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4410, ret 0, client (nil) [ CSSD]2013-05-23 15:38:53.997 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 680 [ CSSD]2013-05-23 15:38:55.617 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:55.720 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:55.720 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:56.809 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:38:56.809 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:38:57.629 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:57.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:57.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:38:57.898 [1191196992] >TRACE: clssgmAllocProc: (0x22da4d0) allocated [ CSSD]2013-05-23 15:38:57.898 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:38:57.899 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22da4d0 - 1,2,3 [ CSSD]2013-05-23 15:38:57.899 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3d50) proc(0x22da4d0) pid(12072) version 11:1:1:4 [ CSSD]2013-05-23 15:38:57.905 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22da4d0), client(1/0x22ef070) [ CSSD]2013-05-23 15:38:57.905 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ef070) [ CSSD]2013-05-23 15:38:57.905 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ef070 with con 0x231da10, requested num -1 [ CSSD]2013-05-23 15:38:57.905 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:57.905 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 681 (0x7f431bfd4490) [ CSSD]2013-05-23 15:38:57.905 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4490 (RPC#681) tag(2a9002c) sent to node 1 [ CSSD]2013-05-23 15:38:57.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02a9002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:57.905 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014046, gin 1, member 0 [ CSSD]2013-05-23 15:38:57.905 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed9a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:38:57.906 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:38:57.906 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:38:57.906 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4490 (RPC#681) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:57.906 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4490, ret 0, client 0x22ef070 member 0x22edca0 [ CSSD]2013-05-23 15:38:57.906 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ef070 [ CSSD]2013-05-23 15:38:57.906 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 681 [ CSSD]2013-05-23 15:38:57.912 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ef070) [ CSSD]2013-05-23 15:38:57.912 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ef070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:57.912 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edca0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:38:57.912 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 682 (0x7f431bfd4510) [ CSSD]2013-05-23 15:38:57.912 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4510 (RPC#682) tag(2aa002c) sent to node 1 [ CSSD]2013-05-23 15:38:57.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02aa002c) incarnation(174631051) [ CSSD]2013-05-23 15:38:57.912 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:38:57.912 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4510 (RPC#682) state 6, flags 0x100 [ CSSD]2013-05-23 15:38:57.912 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4510, ret 0, client 0x22ef070 member 0x22edca0 [ CSSD]2013-05-23 15:38:57.913 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 682 [ CSSD]2013-05-23 15:38:57.913 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22edca0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:38:58.115 [1191196992] >TRACE: clssgmDeadProc: proc 0x22da4d0 [ CSSD]2013-05-23 15:38:58.115 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22da4d0) con(0x22f3d50) skgpid ospid 12072 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:38:59.635 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:38:59.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:38:59.736 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:01.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:01.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:01.761 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:01.812 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:01.813 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:03.018 [1191196992] >TRACE: clssgmAllocProc: (0x22da4d0) allocated [ CSSD]2013-05-23 15:39:03.019 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:39:03.020 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22da4d0 - 1,2,3 [ CSSD]2013-05-23 15:39:03.020 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3d50) proc(0x22da4d0) pid(12035/12035) version 11:1:1:4 [ CSSD]2013-05-23 15:39:03.025 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22da4d0), client(1/0x22ef070) [ CSSD]2013-05-23 15:39:03.025 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ef070) [ CSSD]2013-05-23 15:39:03.025 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:39:03.025 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 15:39:03.269 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22da4d0), client(2/0x231dc50) [ CSSD]2013-05-23 15:39:03.269 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x231dc50) [ CSSD]2013-05-23 15:39:03.269 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:39:03.269 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 15:39:03.692 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:03.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:03.815 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:04.023 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:39:04.023 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 683 (0x7f431bfd4590) [ CSSD]2013-05-23 15:39:04.024 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4590 (RPC#683) tag(2ab002c) sent to node 1 [ CSSD]2013-05-23 15:39:04.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:04.024 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 683 (0x7f431bfd4590) [ CSSD]2013-05-23 15:39:04.024 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4590 (RPC#683) state 6, flags 0x202 [ CSSD]2013-05-23 15:39:04.024 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4590, ret 0, client (nil) [ CSSD]2013-05-23 15:39:04.024 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 683 [ CSSD]2013-05-23 15:39:04.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:04.628 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014047, gin 1, member 0 [ CSSD]2013-05-23 15:39:04.628 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:39:04.628 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:39:04.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:04.635 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:39:04.635 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:39:05.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:05.814 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:05.815 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:39:05.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:05.840 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:07.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:07.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:07.885 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:08.346 [1191196992] >TRACE: clssgmAllocProc: (0x23191d0) allocated [ CSSD]2013-05-23 15:39:08.347 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:39:08.347 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191d0 - 1,2,3 [ CSSD]2013-05-23 15:39:08.347 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23178a0) proc(0x23191d0) pid(12188) version 11:1:1:4 [ CSSD]2013-05-23 15:39:08.364 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23191d0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:39:08.364 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:39:08.364 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x2318fc0, requested num -1 [ CSSD]2013-05-23 15:39:08.364 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:08.364 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 684 (0x7f431bfd4610) [ CSSD]2013-05-23 15:39:08.364 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4610 (RPC#684) tag(2ac002c) sent to node 1 [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ac002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014048, gin 1, member 0 [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4610 (RPC#684) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4610, ret 0, client 0x22ee1b0 member 0x22e43b0 [ CSSD]2013-05-23 15:39:08.364 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:39:08.365 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 684 [ CSSD]2013-05-23 15:39:08.371 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:39:08.371 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:08.371 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e43b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:08.371 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 685 (0x7f431bfd4690) [ CSSD]2013-05-23 15:39:08.371 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4690 (RPC#685) tag(2ad002c) sent to node 1 [ CSSD]2013-05-23 15:39:08.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ad002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:08.371 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:08.371 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4690 (RPC#685) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:08.371 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4690, ret 0, client 0x22ee1b0 member 0x22e43b0 [ CSSD]2013-05-23 15:39:08.371 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 685 [ CSSD]2013-05-23 15:39:08.371 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e43b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:39:08.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:08.411 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014049, gin 1, member 0 [ CSSD]2013-05-23 15:39:08.411 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:39:08.411 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:39:08.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:08.417 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:39:08.417 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:39:08.574 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191d0 [ CSSD]2013-05-23 15:39:08.574 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191d0) con(0x23178a0) skgpid ospid 12188 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:39:09.735 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:09.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:09.917 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:10.817 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:10.818 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:11.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:11.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:11.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:13.147 [1191196992] >TRACE: clssgmAllocProc: (0x23191d0) allocated [ CSSD]2013-05-23 15:39:13.148 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:39:13.149 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191d0 - 1,2,3 [ CSSD]2013-05-23 15:39:13.149 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23178a0) proc(0x23191d0) pid(12222) version 11:1:1:4 [ CSSD]2013-05-23 15:39:13.155 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23191d0), client(1/0x22eda70) [ CSSD]2013-05-23 15:39:13.155 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eda70) [ CSSD]2013-05-23 15:39:13.155 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eda70 with con 0x2318fc0, requested num -1 [ CSSD]2013-05-23 15:39:13.155 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:13.155 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 686 (0x7f431bfd4710) [ CSSD]2013-05-23 15:39:13.155 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4710 (RPC#686) tag(2ae002c) sent to node 1 [ CSSD]2013-05-23 15:39:13.155 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ae002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014050, gin 1, member 0 [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4710 (RPC#686) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4710, ret 0, client 0x22eda70 member 0x22e43b0 [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eda70 [ CSSD]2013-05-23 15:39:13.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 686 [ CSSD]2013-05-23 15:39:13.162 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eda70) [ CSSD]2013-05-23 15:39:13.162 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eda70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:13.162 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e43b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:13.162 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 687 (0x7f431bfd4790) [ CSSD]2013-05-23 15:39:13.162 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4790 (RPC#687) tag(2af002c) sent to node 1 [ CSSD]2013-05-23 15:39:13.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02af002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:13.163 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:13.163 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4790 (RPC#687) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:13.163 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4790, ret 0, client 0x22eda70 member 0x22e43b0 [ CSSD]2013-05-23 15:39:13.163 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 687 [ CSSD]2013-05-23 15:39:13.163 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e43b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:39:13.365 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191d0 [ CSSD]2013-05-23 15:39:13.365 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191d0) con(0x23178a0) skgpid ospid 12222 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:39:13.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:14.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:14.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:14.049 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:39:14.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 688 (0x7f431bfd4810) [ CSSD]2013-05-23 15:39:14.049 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4810 (RPC#688) tag(2b0002c) sent to node 1 [ CSSD]2013-05-23 15:39:14.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:14.050 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 688 (0x7f431bfd4810) [ CSSD]2013-05-23 15:39:14.050 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4810 (RPC#688) state 6, flags 0x202 [ CSSD]2013-05-23 15:39:14.050 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4810, ret 0, client (nil) [ CSSD]2013-05-23 15:39:14.050 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 688 [ CSSD]2013-05-23 15:39:15.798 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:15.821 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:15.821 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:16.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:16.060 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:17.809 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:18.065 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:18.065 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:19.815 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:20.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:20.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:20.824 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:20.824 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:21.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:22.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:22.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:22.602 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:39:22.603 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:39:22.603 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:39:22.604 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:39:22.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:22.685 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014051, gin 1, member 0 [ CSSD]2013-05-23 15:39:22.685 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c120bc0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 15:39:22.685 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 15:39:22.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:22.691 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 15:39:22.691 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:39:23.662 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:23.662 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014052, gin 1, member 0 [ CSSD]2013-05-23 15:39:23.662 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:39:23.662 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:39:23.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:23.668 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:39:23.668 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:39:23.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:24.075 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:39:24.075 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 689 (0x7f431bfd4890) [ CSSD]2013-05-23 15:39:24.075 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4890 (RPC#689) tag(2b1002c) sent to node 1 [ CSSD]2013-05-23 15:39:24.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:24.076 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 689 (0x7f431bfd4890) [ CSSD]2013-05-23 15:39:24.076 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4890 (RPC#689) state 6, flags 0x202 [ CSSD]2013-05-23 15:39:24.076 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4890, ret 0, client (nil) [ CSSD]2013-05-23 15:39:24.076 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 689 [ CSSD]2013-05-23 15:39:24.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:24.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:25.827 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:25.827 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:25.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:26.141 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:26.141 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:27.891 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:28.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:28.146 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:28.396 [1191196992] >TRACE: clssgmAllocProc: (0x2319120) allocated [ CSSD]2013-05-23 15:39:28.397 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:39:28.398 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2319120 - 1,2,3 [ CSSD]2013-05-23 15:39:28.398 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee120) proc(0x2319120) pid(12390) version 11:1:1:4 [ CSSD]2013-05-23 15:39:28.404 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2319120), client(1/0x22dabf0) [ CSSD]2013-05-23 15:39:28.404 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabf0) [ CSSD]2013-05-23 15:39:28.404 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabf0 with con 0x2319380, requested num -1 [ CSSD]2013-05-23 15:39:28.404 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:28.404 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 690 (0x7f431bfd4910) [ CSSD]2013-05-23 15:39:28.404 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4910 (RPC#690) tag(2b2002c) sent to node 1 [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02b2002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014053, gin 1, member 0 [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3ff0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4910 (RPC#690) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4910, ret 0, client 0x22dabf0 member 0x2319000 [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabf0 [ CSSD]2013-05-23 15:39:28.405 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 690 [ CSSD]2013-05-23 15:39:28.411 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabf0) [ CSSD]2013-05-23 15:39:28.411 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabf0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:28.411 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2319000) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:28.411 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 691 (0x7f431bfd4990) [ CSSD]2013-05-23 15:39:28.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4990 (RPC#691) tag(2b3002c) sent to node 1 [ CSSD]2013-05-23 15:39:28.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02b3002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:28.412 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:28.412 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4990 (RPC#691) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:28.412 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4990, ret 0, client 0x22dabf0 member 0x2319000 [ CSSD]2013-05-23 15:39:28.412 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 691 [ CSSD]2013-05-23 15:39:28.412 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2319000) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:39:28.614 [1191196992] >TRACE: clssgmDeadProc: proc 0x2319120 [ CSSD]2013-05-23 15:39:28.614 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2319120) con(0x22ee120) skgpid ospid 12390 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:39:29.829 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:29.830 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:39:29.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:30.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:30.159 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:31.902 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:32.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:32.185 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:33.923 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:34.101 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:39:34.101 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 692 (0x7f431bfd4a10) [ CSSD]2013-05-23 15:39:34.102 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4a10 (RPC#692) tag(2b4002c) sent to node 1 [ CSSD]2013-05-23 15:39:34.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:34.102 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 692 (0x7f431bfd4a10) [ CSSD]2013-05-23 15:39:34.102 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4a10 (RPC#692) state 6, flags 0x202 [ CSSD]2013-05-23 15:39:34.102 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4a10, ret 0, client (nil) [ CSSD]2013-05-23 15:39:34.102 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 692 [ CSSD]2013-05-23 15:39:34.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:34.236 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:34.832 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:34.832 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:35.928 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:36.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:36.241 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:37.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:38.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:38.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:38.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:38.919 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014054, gin 1, member 0 [ CSSD]2013-05-23 15:39:38.919 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:39:38.919 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:39:38.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:38.926 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:39:38.926 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:39:39.835 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:39.835 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:39.946 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:40.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:40.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:41.999 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:42.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:42.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:43.646 [1191196992] >TRACE: clssgmAllocProc: (0x23191d0) allocated [ CSSD]2013-05-23 15:39:43.647 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:39:43.647 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191d0 - 1,2,3 [ CSSD]2013-05-23 15:39:43.647 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23178a0) proc(0x23191d0) pid(12530) version 11:1:1:4 [ CSSD]2013-05-23 15:39:43.652 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23191d0), client(1/0x22eda70) [ CSSD]2013-05-23 15:39:43.653 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eda70) [ CSSD]2013-05-23 15:39:43.653 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eda70 with con 0x2318fc0, requested num -1 [ CSSD]2013-05-23 15:39:43.653 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:43.653 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 693 (0x7f431bfd4a90) [ CSSD]2013-05-23 15:39:43.653 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4a90 (RPC#693) tag(2b5002c) sent to node 1 [ CSSD]2013-05-23 15:39:43.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02b5002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:43.653 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014055, gin 1, member 0 [ CSSD]2013-05-23 15:39:43.653 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:39:43.654 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:39:43.654 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:39:43.654 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4a90 (RPC#693) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:43.654 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4a90, ret 0, client 0x22eda70 member 0x22e43b0 [ CSSD]2013-05-23 15:39:43.654 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eda70 [ CSSD]2013-05-23 15:39:43.654 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 693 [ CSSD]2013-05-23 15:39:43.660 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eda70) [ CSSD]2013-05-23 15:39:43.660 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eda70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:43.660 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e43b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:43.660 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 694 (0x7f431bfd4b10) [ CSSD]2013-05-23 15:39:43.660 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4b10 (RPC#694) tag(2b6002c) sent to node 1 [ CSSD]2013-05-23 15:39:43.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02b6002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:43.661 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:43.661 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4b10 (RPC#694) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:43.661 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4b10, ret 0, client 0x22eda70 member 0x22e43b0 [ CSSD]2013-05-23 15:39:43.661 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 694 [ CSSD]2013-05-23 15:39:43.661 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e43b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:39:43.863 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191d0 [ CSSD]2013-05-23 15:39:43.864 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191d0) con(0x23178a0) skgpid ospid 12530 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:39:44.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:44.125 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:39:44.125 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 695 (0x7f431bfd4b90) [ CSSD]2013-05-23 15:39:44.125 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4b90 (RPC#695) tag(2b7002c) sent to node 1 [ CSSD]2013-05-23 15:39:44.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:44.126 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 695 (0x7f431bfd4b90) [ CSSD]2013-05-23 15:39:44.126 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4b90 (RPC#695) state 6, flags 0x202 [ CSSD]2013-05-23 15:39:44.126 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4b90, ret 0, client (nil) [ CSSD]2013-05-23 15:39:44.126 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 695 [ CSSD]2013-05-23 15:39:44.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:44.317 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:44.838 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:44.839 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:46.029 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:46.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:46.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:48.041 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:48.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:48.348 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:49.842 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:49.842 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:50.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:50.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:50.358 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:52.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:52.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:52.364 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:54.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:54.156 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:39:54.156 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 696 (0x7f431bfd4c10) [ CSSD]2013-05-23 15:39:54.156 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4c10 (RPC#696) tag(2b8002c) sent to node 1 [ CSSD]2013-05-23 15:39:54.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:54.157 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 696 (0x7f431bfd4c10) [ CSSD]2013-05-23 15:39:54.157 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4c10 (RPC#696) state 6, flags 0x202 [ CSSD]2013-05-23 15:39:54.157 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4c10, ret 0, client (nil) [ CSSD]2013-05-23 15:39:54.157 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 696 [ CSSD]2013-05-23 15:39:54.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:54.183 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014056, gin 1, member 0 [ CSSD]2013-05-23 15:39:54.183 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:39:54.183 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:39:54.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:54.190 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:39:54.190 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:39:54.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:54.417 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:54.845 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:54.845 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:39:56.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:56.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:56.423 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:58.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:39:58.435 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:39:58.435 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:39:58.895 [1191196992] >TRACE: clssgmAllocProc: (0x23192b0) allocated [ CSSD]2013-05-23 15:39:58.896 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:39:58.897 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23192b0 - 1,2,3 [ CSSD]2013-05-23 15:39:58.897 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317980) proc(0x23192b0) pid(12693) version 11:1:1:4 [ CSSD]2013-05-23 15:39:58.902 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23192b0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:39:58.902 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:39:58.902 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1b0 with con 0x2319510, requested num -1 [ CSSD]2013-05-23 15:39:58.902 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:58.902 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 697 (0x7f431bfd4c90) [ CSSD]2013-05-23 15:39:58.903 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4c90 (RPC#697) tag(2b9002c) sent to node 1 [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02b9002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014057, gin 1, member 0 [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4180/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4c90 (RPC#697) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4c90, ret 0, client 0x22ee1b0 member 0x2319190 [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1b0 [ CSSD]2013-05-23 15:39:58.903 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 697 [ CSSD]2013-05-23 15:39:58.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:39:58.910 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:58.910 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2319190) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:39:58.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 698 (0x7f431bfd4d10) [ CSSD]2013-05-23 15:39:58.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4d10 (RPC#698) tag(2ba002c) sent to node 1 [ CSSD]2013-05-23 15:39:58.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ba002c) incarnation(174631051) [ CSSD]2013-05-23 15:39:58.910 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:39:58.910 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4d10 (RPC#698) state 6, flags 0x100 [ CSSD]2013-05-23 15:39:58.910 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4d10, ret 0, client 0x22ee1b0 member 0x2319190 [ CSSD]2013-05-23 15:39:58.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 698 [ CSSD]2013-05-23 15:39:58.911 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2319190) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:39:59.113 [1191196992] >TRACE: clssgmDeadProc: proc 0x23192b0 [ CSSD]2013-05-23 15:39:59.113 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23192b0) con(0x2317980) skgpid ospid 12693 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:39:59.848 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:39:59.848 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:00.145 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:00.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:00.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:02.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:02.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:02.446 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:04.205 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:04.207 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:40:04.207 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 699 (0x7f431bfd4d90) [ CSSD]2013-05-23 15:40:04.207 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4d90 (RPC#699) tag(2bb002c) sent to node 1 [ CSSD]2013-05-23 15:40:04.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:04.208 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 699 (0x7f431bfd4d90) [ CSSD]2013-05-23 15:40:04.208 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4d90 (RPC#699) state 6, flags 0x202 [ CSSD]2013-05-23 15:40:04.208 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4d90, ret 0, client (nil) [ CSSD]2013-05-23 15:40:04.208 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 699 [ CSSD]2013-05-23 15:40:04.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:04.478 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:04.852 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:04.852 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:04.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:04.884 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014058, gin 1, member 0 [ CSSD]2013-05-23 15:40:04.884 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:40:04.884 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:40:04.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:04.891 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:40:04.891 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:40:06.212 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:06.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:06.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:08.144 [1191196992] >TRACE: clssgmAllocProc: (0x23191d0) allocated [ CSSD]2013-05-23 15:40:08.145 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:40:08.146 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191d0 - 1,2,3 [ CSSD]2013-05-23 15:40:08.146 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23178a0) proc(0x23191d0) pid(12986) version 11:1:1:4 [ CSSD]2013-05-23 15:40:08.221 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:08.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:08.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:08.614 [1191196992] >TRACE: clssgmAllocProc: (0x2321da0) allocated [ CSSD]2013-05-23 15:40:08.615 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:40:08.616 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2321da0 - 1,2,3 [ CSSD]2013-05-23 15:40:08.616 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2319430) proc(0x2321da0) pid(13099) version 11:1:1:4 [ CSSD]2013-05-23 15:40:08.622 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2321da0), client(1/0x22dabf0) [ CSSD]2013-05-23 15:40:08.622 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabf0) [ CSSD]2013-05-23 15:40:08.622 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabf0 with con 0x2322000, requested num -1 [ CSSD]2013-05-23 15:40:08.622 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:08.622 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 700 (0x7f431bfd4e10) [ CSSD]2013-05-23 15:40:08.622 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4e10 (RPC#700) tag(2bc002c) sent to node 1 [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02bc002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014059, gin 1, member 0 [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23225f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4e10 (RPC#700) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4e10, ret 0, client 0x22dabf0 member 0x2321c70 [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabf0 [ CSSD]2013-05-23 15:40:08.623 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 700 [ CSSD]2013-05-23 15:40:08.629 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabf0) [ CSSD]2013-05-23 15:40:08.629 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabf0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:08.629 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2321c70) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:08.629 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 701 (0x7f431bfd4e90) [ CSSD]2013-05-23 15:40:08.629 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4e90 (RPC#701) tag(2bd002c) sent to node 1 [ CSSD]2013-05-23 15:40:08.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02bd002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:08.629 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:08.629 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4e90 (RPC#701) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:08.629 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4e90, ret 0, client 0x22dabf0 member 0x2321c70 [ CSSD]2013-05-23 15:40:08.629 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 701 [ CSSD]2013-05-23 15:40:08.629 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2321c70) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:40:08.821 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191d0 [ CSSD]2013-05-23 15:40:08.821 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191d0) con(0x23178a0) skgpid ospid 12986 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:08.832 [1191196992] >TRACE: clssgmDeadProc: proc 0x2321da0 [ CSSD]2013-05-23 15:40:08.832 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2321da0) con(0x2319430) skgpid ospid 13099 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:09.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:09.437 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014060, gin 1, member 0 [ CSSD]2013-05-23 15:40:09.437 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:40:09.437 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:40:09.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:09.445 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:40:09.445 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129980) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:40:09.855 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:09.855 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:10.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:10.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:10.532 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:12.300 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:12.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:12.557 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:14.154 [1191196992] >TRACE: clssgmAllocProc: (0x23191d0) allocated [ CSSD]2013-05-23 15:40:14.155 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:40:14.155 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191d0 - 1,2,3 [ CSSD]2013-05-23 15:40:14.155 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23178a0) proc(0x23191d0) pid(13157) version 11:1:1:4 [ CSSD]2013-05-23 15:40:14.161 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23191d0), client(1/0x22eda70) [ CSSD]2013-05-23 15:40:14.161 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eda70) [ CSSD]2013-05-23 15:40:14.161 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eda70 with con 0x2318fc0, requested num -1 [ CSSD]2013-05-23 15:40:14.161 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:14.161 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 702 (0x7f431bfd4f10) [ CSSD]2013-05-23 15:40:14.161 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4f10 (RPC#702) tag(2be002c) sent to node 1 [ CSSD]2013-05-23 15:40:14.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02be002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:14.161 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014061, gin 1, member 0 [ CSSD]2013-05-23 15:40:14.161 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:40:14.162 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:40:14.162 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:40:14.162 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4f10 (RPC#702) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:14.162 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4f10, ret 0, client 0x22eda70 member 0x22e43b0 [ CSSD]2013-05-23 15:40:14.162 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eda70 [ CSSD]2013-05-23 15:40:14.162 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 702 [ CSSD]2013-05-23 15:40:14.168 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eda70) [ CSSD]2013-05-23 15:40:14.168 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eda70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:14.168 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e43b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:14.168 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 703 (0x7f431bfd4f90) [ CSSD]2013-05-23 15:40:14.168 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4f90 (RPC#703) tag(2bf002c) sent to node 1 [ CSSD]2013-05-23 15:40:14.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02bf002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:14.168 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:14.168 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4f90 (RPC#703) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:14.168 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4f90, ret 0, client 0x22eda70 member 0x22e43b0 [ CSSD]2013-05-23 15:40:14.169 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 703 [ CSSD]2013-05-23 15:40:14.169 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e43b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:40:14.295 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:40:14.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 704 (0x7f431bfd5010) [ CSSD]2013-05-23 15:40:14.295 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5010 (RPC#704) tag(2c0002c) sent to node 1 [ CSSD]2013-05-23 15:40:14.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:14.296 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 704 (0x7f431bfd5010) [ CSSD]2013-05-23 15:40:14.296 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5010 (RPC#704) state 6, flags 0x202 [ CSSD]2013-05-23 15:40:14.296 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5010, ret 0, client (nil) [ CSSD]2013-05-23 15:40:14.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 704 [ CSSD]2013-05-23 15:40:14.325 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:14.371 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191d0 [ CSSD]2013-05-23 15:40:14.371 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191d0) con(0x23178a0) skgpid ospid 13157 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:14.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:14.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:14.858 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:14.858 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:16.331 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:16.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:16.673 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:18.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:18.543 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c12a550) [ CSSD]2013-05-23 15:40:18.543 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c12a550), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:40:18.543 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c12a550), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:40:18.543 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 15:40:18.553 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f3b10) [ CSSD]2013-05-23 15:40:18.553 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f3b10), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:40:18.553 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f3b10), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:40:18.553 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 15:40:18.553 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7970 [ CSSD]2013-05-23 15:40:18.553 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7970) con(0x22dacd0) skgpid 7675 ospid 7675 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:18.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:18.726 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:19.862 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:19.862 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:20.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:20.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:20.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:22.400 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:22.555 [1191196992] >TRACE: clssgmAllocProc: (0x22ff460) allocated [ CSSD]2013-05-23 15:40:22.555 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:40:22.556 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ff460 - 1,2,3 [ CSSD]2013-05-23 15:40:22.556 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22eeea0) proc(0x22ff460) pid(13244/13244) version 11:1:1:4 [ CSSD]2013-05-23 15:40:22.572 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ff460), client(1/0x22f3be0) [ CSSD]2013-05-23 15:40:22.572 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f3be0) [ CSSD]2013-05-23 15:40:22.572 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:40:22.572 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 15:40:22.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:22.738 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:23.026 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ff460), client(2/0x22dabf0) [ CSSD]2013-05-23 15:40:23.026 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22dabf0) [ CSSD]2013-05-23 15:40:23.026 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:40:23.026 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 15:40:24.322 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:40:24.323 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 705 (0x7f431bfd5090) [ CSSD]2013-05-23 15:40:24.323 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5090 (RPC#705) tag(2c1002c) sent to node 1 [ CSSD]2013-05-23 15:40:24.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:24.323 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 705 (0x7f431bfd5090) [ CSSD]2013-05-23 15:40:24.323 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5090 (RPC#705) state 6, flags 0x202 [ CSSD]2013-05-23 15:40:24.323 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5090, ret 0, client (nil) [ CSSD]2013-05-23 15:40:24.324 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 705 [ CSSD]2013-05-23 15:40:24.406 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:24.688 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:24.688 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014062, gin 1, member 0 [ CSSD]2013-05-23 15:40:24.688 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:40:24.688 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:40:24.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:24.695 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:40:24.695 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:40:24.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:24.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:24.865 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:24.865 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:26.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:26.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:26.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:28.422 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:28.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:28.846 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:29.403 [1191196992] >TRACE: clssgmAllocProc: (0x22e3c50) allocated [ CSSD]2013-05-23 15:40:29.404 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:40:29.404 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e3c50 - 1,2,3 [ CSSD]2013-05-23 15:40:29.404 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x22e3c50) pid(13313) version 11:1:1:4 [ CSSD]2013-05-23 15:40:29.410 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e3c50), client(1/0x22e42f0) [ CSSD]2013-05-23 15:40:29.410 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e42f0) [ CSSD]2013-05-23 15:40:29.410 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e42f0 with con 0x22e3de0, requested num -1 [ CSSD]2013-05-23 15:40:29.410 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:29.410 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 706 (0x7f431bfd5110) [ CSSD]2013-05-23 15:40:29.410 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5110 (RPC#706) tag(2c2002c) sent to node 1 [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c2002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014063, gin 1, member 0 [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4460/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5110 (RPC#706) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5110, ret 0, client 0x22e42f0 member 0x23193e0 [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e42f0 [ CSSD]2013-05-23 15:40:29.411 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 706 [ CSSD]2013-05-23 15:40:29.417 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e42f0) [ CSSD]2013-05-23 15:40:29.417 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e42f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:29.417 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23193e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:29.417 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 707 (0x7f431bfd5190) [ CSSD]2013-05-23 15:40:29.417 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5190 (RPC#707) tag(2c3002c) sent to node 1 [ CSSD]2013-05-23 15:40:29.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02c3002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:29.418 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:29.418 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5190 (RPC#707) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:29.418 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5190, ret 0, client 0x22e42f0 member 0x23193e0 [ CSSD]2013-05-23 15:40:29.418 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 707 [ CSSD]2013-05-23 15:40:29.418 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23193e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:40:29.620 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e3c50 [ CSSD]2013-05-23 15:40:29.620 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e3c50) con(0x22dacd0) skgpid ospid 13313 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:29.868 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:29.868 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:30.446 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:30.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:30.851 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:32.501 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:32.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:32.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:34.354 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:40:34.354 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 708 (0x7f431bfd5210) [ CSSD]2013-05-23 15:40:34.354 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5210 (RPC#708) tag(2c4002c) sent to node 1 [ CSSD]2013-05-23 15:40:34.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:34.355 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 708 (0x7f431bfd5210) [ CSSD]2013-05-23 15:40:34.355 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5210 (RPC#708) state 6, flags 0x202 [ CSSD]2013-05-23 15:40:34.355 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5210, ret 0, client (nil) [ CSSD]2013-05-23 15:40:34.355 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 708 [ CSSD]2013-05-23 15:40:34.507 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:34.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:34.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:34.872 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:34.872 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:36.532 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:36.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:36.875 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:38.542 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:38.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:38.927 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:39.875 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:39.875 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:39.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:39.938 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014064, gin 1, member 0 [ CSSD]2013-05-23 15:40:39.938 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:40:39.938 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:40:39.945 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:39.945 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:40:39.945 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:40:40.548 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:40.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:40.933 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:42.601 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:42.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:42.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:44.384 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:40:44.384 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 709 (0x7f431bfd5290) [ CSSD]2013-05-23 15:40:44.384 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5290 (RPC#709) tag(2c5002c) sent to node 1 [ CSSD]2013-05-23 15:40:44.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:44.385 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 709 (0x7f431bfd5290) [ CSSD]2013-05-23 15:40:44.385 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5290 (RPC#709) state 6, flags 0x202 [ CSSD]2013-05-23 15:40:44.385 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5290, ret 0, client (nil) [ CSSD]2013-05-23 15:40:44.385 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 709 [ CSSD]2013-05-23 15:40:44.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:44.651 [1191196992] >TRACE: clssgmAllocProc: (0x22e3c50) allocated [ CSSD]2013-05-23 15:40:44.652 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:40:44.653 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e3c50 - 1,2,3 [ CSSD]2013-05-23 15:40:44.653 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317c80) proc(0x22e3c50) pid(13462) version 11:1:1:4 [ CSSD]2013-05-23 15:40:44.658 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e3c50), client(1/0x22e43a0) [ CSSD]2013-05-23 15:40:44.658 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e43a0) [ CSSD]2013-05-23 15:40:44.658 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e43a0 with con 0x22e3e90, requested num -1 [ CSSD]2013-05-23 15:40:44.658 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:44.658 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 710 (0x7f431bfd5310) [ CSSD]2013-05-23 15:40:44.659 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5310 (RPC#710) tag(2c6002c) sent to node 1 [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c6002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014065, gin 1, member 0 [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4510/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5310 (RPC#710) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5310, ret 0, client 0x22e43a0 member 0x2319490 [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e43a0 [ CSSD]2013-05-23 15:40:44.659 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 710 [ CSSD]2013-05-23 15:40:44.666 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e43a0) [ CSSD]2013-05-23 15:40:44.666 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e43a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:44.666 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2319490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:44.666 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 711 (0x7f431bfd5390) [ CSSD]2013-05-23 15:40:44.666 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5390 (RPC#711) tag(2c7002c) sent to node 1 [ CSSD]2013-05-23 15:40:44.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02c7002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:44.666 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:44.666 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5390 (RPC#711) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:44.666 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5390, ret 0, client 0x22e43a0 member 0x2319490 [ CSSD]2013-05-23 15:40:44.667 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 711 [ CSSD]2013-05-23 15:40:44.667 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2319490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:40:44.869 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e3c50 [ CSSD]2013-05-23 15:40:44.869 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e3c50) con(0x2317c80) skgpid ospid 13462 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:44.877 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:44.878 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:44.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:44.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:46.632 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:46.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:46.975 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:48.644 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:49.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:49.028 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:49.881 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:49.881 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:50.669 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:51.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:51.034 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:52.712 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:53.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:53.040 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:54.411 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:40:54.411 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 712 (0x7f431bfd5410) [ CSSD]2013-05-23 15:40:54.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5410 (RPC#712) tag(2c8002c) sent to node 1 [ CSSD]2013-05-23 15:40:54.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:54.412 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 712 (0x7f431bfd5410) [ CSSD]2013-05-23 15:40:54.412 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5410 (RPC#712) state 6, flags 0x202 [ CSSD]2013-05-23 15:40:54.412 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5410, ret 0, client (nil) [ CSSD]2013-05-23 15:40:54.412 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 712 [ CSSD]2013-05-23 15:40:54.717 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:54.884 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:54.884 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:55.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:55.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:55.188 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:55.188 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014066, gin 1, member 0 [ CSSD]2013-05-23 15:40:55.188 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:40:55.188 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:40:55.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:55.195 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:40:55.195 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:40:55.260 [1191196992] >TRACE: clssgmAllocProc: (0x23193d0) allocated [ CSSD]2013-05-23 15:40:55.261 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:40:55.262 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23193d0 - 1,2,3 [ CSSD]2013-05-23 15:40:55.262 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x23193d0) pid(13608) version 11:1:1:4 [ CSSD]2013-05-23 15:40:55.268 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23193d0), client(1/0x22d7350) [ CSSD]2013-05-23 15:40:55.268 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7350) [ CSSD]2013-05-23 15:40:55.268 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7350 with con 0x22e3dd0, requested num -1 [ CSSD]2013-05-23 15:40:55.268 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:55.268 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 713 (0x7f431bfd5490) [ CSSD]2013-05-23 15:40:55.268 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5490 (RPC#713) tag(2c9002c) sent to node 1 [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c9002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014067, gin 1, member 0 [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e43b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5490 (RPC#713) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5490, ret 0, client 0x22d7350 member 0x22e46b0 [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7350 [ CSSD]2013-05-23 15:40:55.269 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 713 [ CSSD]2013-05-23 15:40:55.276 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7350) [ CSSD]2013-05-23 15:40:55.276 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7350), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:55.276 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e46b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:55.276 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 714 (0x7f431bfd5510) [ CSSD]2013-05-23 15:40:55.276 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5510 (RPC#714) tag(2ca002c) sent to node 1 [ CSSD]2013-05-23 15:40:55.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ca002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:55.276 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:55.276 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5510 (RPC#714) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:55.276 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5510, ret 0, client 0x22d7350 member 0x22e46b0 [ CSSD]2013-05-23 15:40:55.276 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 714 [ CSSD]2013-05-23 15:40:55.276 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e46b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:40:55.479 [1191196992] >TRACE: clssgmDeadProc: proc 0x23193d0 [ CSSD]2013-05-23 15:40:55.479 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23193d0) con(0x22dacd0) skgpid ospid 13608 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:40:56.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:57.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:57.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:58.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:40:59.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:40:59.129 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:40:59.887 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:40:59.887 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:40:59.910 [1191196992] >TRACE: clssgmAllocProc: (0x23193d0) allocated [ CSSD]2013-05-23 15:40:59.911 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:40:59.922 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23193d0 - 1,2,3 [ CSSD]2013-05-23 15:40:59.922 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x23193d0) pid(13639) version 11:1:1:4 [ CSSD]2013-05-23 15:40:59.928 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23193d0), client(1/0x22e42e0) [ CSSD]2013-05-23 15:40:59.928 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e42e0) [ CSSD]2013-05-23 15:40:59.928 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e42e0 with con 0x22e3dd0, requested num -1 [ CSSD]2013-05-23 15:40:59.928 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:59.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 715 (0x7f431bfd5590) [ CSSD]2013-05-23 15:40:59.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5590 (RPC#715) tag(2cb002c) sent to node 1 [ CSSD]2013-05-23 15:40:59.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02cb002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:59.928 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014068, gin 1, member 0 [ CSSD]2013-05-23 15:40:59.928 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4450/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:40:59.929 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:40:59.929 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:40:59.929 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5590 (RPC#715) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:59.929 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5590, ret 0, client 0x22e42e0 member 0x22e4750 [ CSSD]2013-05-23 15:40:59.929 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e42e0 [ CSSD]2013-05-23 15:40:59.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 715 [ CSSD]2013-05-23 15:40:59.935 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e42e0) [ CSSD]2013-05-23 15:40:59.935 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e42e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:59.935 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:40:59.935 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 716 (0x7f431bfd5610) [ CSSD]2013-05-23 15:40:59.935 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5610 (RPC#716) tag(2cc002c) sent to node 1 [ CSSD]2013-05-23 15:40:59.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02cc002c) incarnation(174631051) [ CSSD]2013-05-23 15:40:59.936 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:40:59.936 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5610 (RPC#716) state 6, flags 0x100 [ CSSD]2013-05-23 15:40:59.936 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5610, ret 0, client 0x22e42e0 member 0x22e4750 [ CSSD]2013-05-23 15:40:59.936 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 716 [ CSSD]2013-05-23 15:40:59.936 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:41:00.139 [1191196992] >TRACE: clssgmDeadProc: proc 0x23193d0 [ CSSD]2013-05-23 15:41:00.139 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23193d0) con(0x22dacd0) skgpid ospid 13639 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:00.760 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:01.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:01.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:02.812 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:03.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:03.140 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:04.444 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:41:04.444 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 717 (0x7f431bfd5690) [ CSSD]2013-05-23 15:41:04.444 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5690 (RPC#717) tag(2cd002c) sent to node 1 [ CSSD]2013-05-23 15:41:04.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:04.445 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 717 (0x7f431bfd5690) [ CSSD]2013-05-23 15:41:04.445 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5690 (RPC#717) state 6, flags 0x202 [ CSSD]2013-05-23 15:41:04.445 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5690, ret 0, client (nil) [ CSSD]2013-05-23 15:41:04.445 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 717 [ CSSD]2013-05-23 15:41:04.818 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:04.890 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:04.890 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:05.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:05.147 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014069, gin 1, member 0 [ CSSD]2013-05-23 15:41:05.147 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:41:05.147 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:41:05.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:05.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:05.154 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:05.154 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:41:05.154 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:41:06.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:07.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:07.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:08.836 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:08.881 [1191196992] >TRACE: clssgmAllocProc: (0x22e3c50) allocated [ CSSD]2013-05-23 15:41:08.883 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:41:08.883 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e3c50 - 1,2,3 [ CSSD]2013-05-23 15:41:08.883 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317c80) proc(0x22e3c50) pid(13748) version 11:1:1:4 [ CSSD]2013-05-23 15:41:08.889 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e3c50), client(1/0x22e43a0) [ CSSD]2013-05-23 15:41:08.889 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e43a0) [ CSSD]2013-05-23 15:41:08.889 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e43a0 with con 0x22e3e90, requested num -1 [ CSSD]2013-05-23 15:41:08.889 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:08.890 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 718 (0x7f431bfd5710) [ CSSD]2013-05-23 15:41:08.890 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5710 (RPC#718) tag(2ce002c) sent to node 1 [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ce002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014070, gin 1, member 0 [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4510/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5710 (RPC#718) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5710, ret 0, client 0x22e43a0 member 0x2319490 [ CSSD]2013-05-23 15:41:08.890 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e43a0 [ CSSD]2013-05-23 15:41:08.891 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 718 [ CSSD]2013-05-23 15:41:08.897 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e43a0) [ CSSD]2013-05-23 15:41:08.897 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e43a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:08.897 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2319490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:08.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 719 (0x7f431bfd5790) [ CSSD]2013-05-23 15:41:08.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5790 (RPC#719) tag(2cf002c) sent to node 1 [ CSSD]2013-05-23 15:41:08.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02cf002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:08.898 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:08.898 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5790 (RPC#719) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:08.898 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5790, ret 0, client 0x22e43a0 member 0x2319490 [ CSSD]2013-05-23 15:41:08.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 719 [ CSSD]2013-05-23 15:41:08.898 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2319490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:41:09.100 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e3c50 [ CSSD]2013-05-23 15:41:09.100 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e3c50) con(0x2317c80) skgpid ospid 13748 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:09.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:09.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:09.893 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:09.893 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:10.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:10.437 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014071, gin 1, member 0 [ CSSD]2013-05-23 15:41:10.437 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:41:10.437 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:41:10.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:10.444 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:41:10.444 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:41:10.861 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:11.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:11.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:12.932 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:13.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:13.279 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:13.895 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:13.896 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:41:14.470 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:41:14.470 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 720 (0x7f431bfd5810) [ CSSD]2013-05-23 15:41:14.470 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5810 (RPC#720) tag(2d0002c) sent to node 1 [ CSSD]2013-05-23 15:41:14.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:14.471 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 720 (0x7f431bfd5810) [ CSSD]2013-05-23 15:41:14.471 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5810 (RPC#720) state 6, flags 0x202 [ CSSD]2013-05-23 15:41:14.471 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5810, ret 0, client (nil) [ CSSD]2013-05-23 15:41:14.471 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 720 [ CSSD]2013-05-23 15:41:14.801 [1191196992] >TRACE: clssgmAllocProc: (0x22e3c50) allocated [ CSSD]2013-05-23 15:41:14.802 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:41:14.803 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e3c50 - 1,2,3 [ CSSD]2013-05-23 15:41:14.803 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x22e3c50) pid(13851) version 11:1:1:4 [ CSSD]2013-05-23 15:41:14.809 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e3c50), client(1/0x22e4380) [ CSSD]2013-05-23 15:41:14.809 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e4380) [ CSSD]2013-05-23 15:41:14.809 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22e4380 with con 0x22e3e70, requested num -1 [ CSSD]2013-05-23 15:41:14.809 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:41:14.809 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 721 (0x7f431bfd5890) [ CSSD]2013-05-23 15:41:14.809 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5890 (RPC#721) tag(2d1002c) sent to node 1 [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02d1002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014072, gin 1, member 0 [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e44f0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5890 (RPC#721) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5890, ret 0, client 0x22e4380 member 0x23193e0 [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e4380 [ CSSD]2013-05-23 15:41:14.810 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 721 [ CSSD]2013-05-23 15:41:14.816 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e4380) [ CSSD]2013-05-23 15:41:14.816 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e4380), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:41:14.816 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23193e0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:41:14.816 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 722 (0x7f431bfd5910) [ CSSD]2013-05-23 15:41:14.816 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5910 (RPC#722) tag(2d2002c) sent to node 1 [ CSSD]2013-05-23 15:41:14.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d2002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:14.817 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:41:14.817 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5910 (RPC#722) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:14.817 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5910, ret 0, client 0x22e4380 member 0x23193e0 [ CSSD]2013-05-23 15:41:14.817 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 722 [ CSSD]2013-05-23 15:41:14.817 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x23193e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:41:14.928 [1191196992] >TRACE: clssgmAllocProc: (0x22e53c0) allocated [ CSSD]2013-05-23 15:41:14.929 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:41:14.930 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e53c0 - 1,2,3 [ CSSD]2013-05-23 15:41:14.930 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e4520) proc(0x22e53c0) pid(13856) version 11:1:1:4 [ CSSD]2013-05-23 15:41:14.938 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:14.944 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:41:14.945 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:41:14.947 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e53c0 [ CSSD]2013-05-23 15:41:14.947 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e53c0) con(0x22e4520) skgpid ospid 13856 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:15.019 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e3c50 [ CSSD]2013-05-23 15:41:15.019 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e3c50) con(0x22dacd0) skgpid ospid 13851 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:15.171 [1191196992] >TRACE: clssgmAllocProc: (0x2322370) allocated [ CSSD]2013-05-23 15:41:15.172 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:41:15.172 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322370 - 1,2,3 [ CSSD]2013-05-23 15:41:15.172 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x2322370) pid(13868) version 11:1:1:4 [ CSSD]2013-05-23 15:41:15.178 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2322370), client(1/0x23231b0) [ CSSD]2013-05-23 15:41:15.178 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23231b0) [ CSSD]2013-05-23 15:41:15.178 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23231b0 with con 0x2323030, requested num -1 [ CSSD]2013-05-23 15:41:15.178 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:15.178 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 723 (0x7f431bfd5990) [ CSSD]2013-05-23 15:41:15.178 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5990 (RPC#723) tag(2d3002c) sent to node 1 [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02d3002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014073, gin 1, member 0 [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4110/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5990 (RPC#723) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5990, ret 0, client 0x23231b0 member 0x2322d50 [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23231b0 [ CSSD]2013-05-23 15:41:15.179 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 723 [ CSSD]2013-05-23 15:41:15.185 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23231b0) [ CSSD]2013-05-23 15:41:15.185 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23231b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:15.185 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2322d50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:15.185 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 724 (0x7f431bfd5a10) [ CSSD]2013-05-23 15:41:15.185 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5a10 (RPC#724) tag(2d4002c) sent to node 1 [ CSSD]2013-05-23 15:41:15.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d4002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:15.186 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:15.186 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5a10 (RPC#724) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:15.186 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5a10, ret 0, client 0x23231b0 member 0x2322d50 [ CSSD]2013-05-23 15:41:15.186 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 724 [ CSSD]2013-05-23 15:41:15.186 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2322d50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:41:15.332 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:15.332 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:15.388 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322370 [ CSSD]2013-05-23 15:41:15.388 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322370) con(0x22dacd0) skgpid ospid 13868 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:16.944 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:17.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:17.356 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:18.899 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:18.899 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:18.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:19.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:19.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:20.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:21.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:21.374 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:22.849 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f4820) [ CSSD]2013-05-23 15:41:22.849 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f4820), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:41:22.849 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f4820), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:41:22.849 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 15:41:22.853 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f4340) [ CSSD]2013-05-23 15:41:22.854 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f4340), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:41:22.854 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f4340), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:41:22.854 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 15:41:22.854 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2de0 [ CSSD]2013-05-23 15:41:22.854 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2de0) con(0x205e170) skgpid 8380 ospid 8380 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:23.013 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:23.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:23.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:23.902 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:23.903 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:24.498 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:41:24.498 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 725 (0x7f431bfd5a90) [ CSSD]2013-05-23 15:41:24.499 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5a90 (RPC#725) tag(2d5002c) sent to node 1 [ CSSD]2013-05-23 15:41:24.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:24.499 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 725 (0x7f431bfd5a90) [ CSSD]2013-05-23 15:41:24.499 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5a90 (RPC#725) state 6, flags 0x202 [ CSSD]2013-05-23 15:41:24.499 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5a90, ret 0, client (nil) [ CSSD]2013-05-23 15:41:24.499 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 725 [ CSSD]2013-05-23 15:41:25.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:25.433 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:25.433 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:25.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:25.687 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014074, gin 1, member 0 [ CSSD]2013-05-23 15:41:25.687 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:41:25.687 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:41:25.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:25.693 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:41:25.693 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:41:27.050 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:27.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:27.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:28.906 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:28.906 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:29.056 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:29.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:29.449 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:30.421 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:41:30.422 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:41:30.423 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:41:30.423 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c1213e0) pid(14052) version 11:1:1:4 [ CSSD]2013-05-23 15:41:30.440 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:41:30.440 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:41:30.440 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:41:30.440 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:30.440 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 726 (0x7f431bfd5b10) [ CSSD]2013-05-23 15:41:30.440 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5b10 (RPC#726) tag(2d6002c) sent to node 1 [ CSSD]2013-05-23 15:41:30.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02d6002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:30.440 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014075, gin 1, member 0 [ CSSD]2013-05-23 15:41:30.440 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:41:30.441 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:41:30.441 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:41:30.441 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5b10 (RPC#726) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:30.441 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5b10, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:41:30.441 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:41:30.441 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 726 [ CSSD]2013-05-23 15:41:30.448 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:41:30.448 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:30.448 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:30.448 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 727 (0x7f431bfd5b90) [ CSSD]2013-05-23 15:41:30.448 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5b90 (RPC#727) tag(2d7002c) sent to node 1 [ CSSD]2013-05-23 15:41:30.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d7002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:30.448 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:30.448 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5b90 (RPC#727) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:30.448 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5b90, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:41:30.448 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 727 [ CSSD]2013-05-23 15:41:30.448 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:41:30.651 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:41:30.651 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c12a550) skgpid ospid 14052 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:31.062 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:31.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:31.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:32.909 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:32.909 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:41:33.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:33.538 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:33.538 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:34.525 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:41:34.525 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 728 (0x7f431bfd5c10) [ CSSD]2013-05-23 15:41:34.525 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5c10 (RPC#728) tag(2d8002c) sent to node 1 [ CSSD]2013-05-23 15:41:34.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:34.526 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 728 (0x7f431bfd5c10) [ CSSD]2013-05-23 15:41:34.526 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5c10 (RPC#728) state 6, flags 0x202 [ CSSD]2013-05-23 15:41:34.526 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5c10, ret 0, client (nil) [ CSSD]2013-05-23 15:41:34.526 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 728 [ CSSD]2013-05-23 15:41:35.120 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:35.544 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:35.544 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:37.132 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:37.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:37.549 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:37.913 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:37.913 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:39.138 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:39.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:39.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:40.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:40.956 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014076, gin 1, member 0 [ CSSD]2013-05-23 15:41:40.956 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:41:40.956 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:41:40.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:40.962 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:41:40.962 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:41:41.144 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:41.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:41.566 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:42.916 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:42.916 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:43.154 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:43.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:43.578 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:44.554 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:41:44.554 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 729 (0x7f431bfd5c90) [ CSSD]2013-05-23 15:41:44.554 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5c90 (RPC#729) tag(2d9002c) sent to node 1 [ CSSD]2013-05-23 15:41:44.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:44.555 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 729 (0x7f431bfd5c90) [ CSSD]2013-05-23 15:41:44.555 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5c90 (RPC#729) state 6, flags 0x202 [ CSSD]2013-05-23 15:41:44.555 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5c90, ret 0, client (nil) [ CSSD]2013-05-23 15:41:44.555 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 729 [ CSSD]2013-05-23 15:41:45.179 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:45.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:45.584 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:45.682 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c121440) allocated [ CSSD]2013-05-23 15:41:45.683 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:41:45.684 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c121440 - 1,2,3 [ CSSD]2013-05-23 15:41:45.684 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c121440) pid(14208) version 11:1:1:4 [ CSSD]2013-05-23 15:41:45.689 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c121440), client(1/0x22f4150) [ CSSD]2013-05-23 15:41:45.689 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4150) [ CSSD]2013-05-23 15:41:45.689 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4150 with con 0x22dacd0, requested num -1 [ CSSD]2013-05-23 15:41:45.689 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:45.689 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 730 (0x7f431bfd5d10) [ CSSD]2013-05-23 15:41:45.690 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5d10 (RPC#730) tag(2da002c) sent to node 1 [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02da002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014077, gin 1, member 0 [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5d10 (RPC#730) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5d10, ret 0, client 0x22f4150 member 0x22f3e00 [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4150 [ CSSD]2013-05-23 15:41:45.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 730 [ CSSD]2013-05-23 15:41:45.697 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4150) [ CSSD]2013-05-23 15:41:45.697 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4150), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:45.697 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3e00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:41:45.697 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 731 (0x7f431bfd5d90) [ CSSD]2013-05-23 15:41:45.697 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5d90 (RPC#731) tag(2db002c) sent to node 1 [ CSSD]2013-05-23 15:41:45.697 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02db002c) incarnation(174631051) [ CSSD]2013-05-23 15:41:45.697 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:41:45.697 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5d90 (RPC#731) state 6, flags 0x100 [ CSSD]2013-05-23 15:41:45.697 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5d90, ret 0, client 0x22f4150 member 0x22f3e00 [ CSSD]2013-05-23 15:41:45.697 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 731 [ CSSD]2013-05-23 15:41:45.697 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3e00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:41:45.900 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c121440 [ CSSD]2013-05-23 15:41:45.900 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c121440) con(0x7f431c11ab50) skgpid ospid 14208 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:41:47.212 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:47.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:47.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:47.919 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:47.920 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:49.238 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:49.641 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:49.641 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:51.244 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:51.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:51.647 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:51.922 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:51.922 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:41:53.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:53.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:53.659 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:54.583 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:41:54.583 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 732 (0x7f431bfd5e10) [ CSSD]2013-05-23 15:41:54.583 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5e10 (RPC#732) tag(2dc002c) sent to node 1 [ CSSD]2013-05-23 15:41:54.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:54.584 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 732 (0x7f431bfd5e10) [ CSSD]2013-05-23 15:41:54.584 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5e10 (RPC#732) state 6, flags 0x202 [ CSSD]2013-05-23 15:41:54.584 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5e10, ret 0, client (nil) [ CSSD]2013-05-23 15:41:54.584 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 732 [ CSSD]2013-05-23 15:41:55.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:55.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:55.664 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:56.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:56.214 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014078, gin 1, member 0 [ CSSD]2013-05-23 15:41:56.214 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:41:56.214 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:41:56.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:56.220 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:41:56.220 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:41:56.925 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:41:56.925 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:41:57.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:57.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:57.670 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:41:59.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:41:59.681 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:41:59.681 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:00.931 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:00.932 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:42:00.932 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:00.932 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c1213e0) pid(14355) version 11:1:1:4 [ CSSD]2013-05-23 15:42:00.938 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:42:00.938 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:00.938 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 15:42:00.938 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:00.938 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 733 (0x7f431bfd5e90) [ CSSD]2013-05-23 15:42:00.938 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5e90 (RPC#733) tag(2dd002c) sent to node 1 [ CSSD]2013-05-23 15:42:00.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02dd002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:00.938 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014079, gin 1, member 0 [ CSSD]2013-05-23 15:42:00.938 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:42:00.938 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:00.939 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:42:00.939 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5e90 (RPC#733) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:00.939 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5e90, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:00.939 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:42:00.939 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 733 [ CSSD]2013-05-23 15:42:00.945 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:00.945 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:00.945 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:00.945 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 734 (0x7f431bfd5f10) [ CSSD]2013-05-23 15:42:00.945 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5f10 (RPC#734) tag(2de002c) sent to node 1 [ CSSD]2013-05-23 15:42:00.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02de002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:00.946 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:00.946 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5f10 (RPC#734) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:00.946 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5f10, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:00.946 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 734 [ CSSD]2013-05-23 15:42:00.946 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:01.148 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:01.148 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c11ab50) skgpid ospid 14355 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:01.324 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:01.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:01.727 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:01.929 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:01.929 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:03.334 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:03.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:03.739 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:04.627 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:42:04.627 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 735 (0x7f431bfd5f90) [ CSSD]2013-05-23 15:42:04.627 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5f90 (RPC#735) tag(2df002c) sent to node 1 [ CSSD]2013-05-23 15:42:04.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:04.628 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 735 (0x7f431bfd5f90) [ CSSD]2013-05-23 15:42:04.628 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5f90 (RPC#735) state 6, flags 0x202 [ CSSD]2013-05-23 15:42:04.628 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5f90, ret 0, client (nil) [ CSSD]2013-05-23 15:42:04.628 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 735 [ CSSD]2013-05-23 15:42:05.291 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:05.292 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:42:05.292 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:05.292 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c1213e0) pid(14424) version 11:1:1:4 [ CSSD]2013-05-23 15:42:05.298 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:42:05.299 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:05.299 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:42:05.299 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:05.299 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 736 (0x7f431bfd6010) [ CSSD]2013-05-23 15:42:05.299 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6010 (RPC#736) tag(2e0002c) sent to node 1 [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e0002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014080, gin 1, member 0 [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6010 (RPC#736) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6010, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:05.299 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:42:05.300 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 736 [ CSSD]2013-05-23 15:42:05.306 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:05.306 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:05.306 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:05.306 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 737 (0x7f431bfd6090) [ CSSD]2013-05-23 15:42:05.306 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6090 (RPC#737) tag(2e1002c) sent to node 1 [ CSSD]2013-05-23 15:42:05.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e1002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:05.307 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:05.307 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6090 (RPC#737) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:05.307 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6090, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:05.307 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 737 [ CSSD]2013-05-23 15:42:05.307 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:05.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:05.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:05.408 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014081, gin 1, member 0 [ CSSD]2013-05-23 15:42:05.408 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c090ff0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:42:05.408 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:42:05.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:05.416 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:42:05.416 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c090dc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:42:05.509 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:05.509 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c12a550) skgpid ospid 14424 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:05.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:05.745 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:06.932 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:06.932 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:07.362 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:07.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:07.750 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:09.131 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:09.132 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:42:09.133 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:09.133 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c1213e0) pid(14459) version 11:1:1:4 [ CSSD]2013-05-23 15:42:09.138 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:42:09.138 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:09.138 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:42:09.138 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:09.138 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 738 (0x7f431bfd6110) [ CSSD]2013-05-23 15:42:09.139 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6110 (RPC#738) tag(2e2002c) sent to node 1 [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e2002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014082, gin 1, member 0 [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6110 (RPC#738) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6110, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:42:09.139 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 738 [ CSSD]2013-05-23 15:42:09.147 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:09.147 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:09.147 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:09.147 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 739 (0x7f431bfd6190) [ CSSD]2013-05-23 15:42:09.147 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6190 (RPC#739) tag(2e3002c) sent to node 1 [ CSSD]2013-05-23 15:42:09.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e3002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:09.147 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:09.147 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6190 (RPC#739) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:09.147 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6190, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:09.148 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 739 [ CSSD]2013-05-23 15:42:09.148 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:09.350 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:09.350 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c12a550) skgpid ospid 14459 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:09.367 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:09.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:09.761 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:10.934 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:10.935 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:42:11.414 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:11.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:11.472 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014083, gin 1, member 0 [ CSSD]2013-05-23 15:42:11.472 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:42:11.472 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:42:11.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:11.479 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:42:11.479 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:42:11.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:11.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:13.063 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:13.064 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:42:13.065 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:13.065 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c1213e0) pid(14617) version 11:1:1:4 [ CSSD]2013-05-23 15:42:13.208 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136fd0) allocated [ CSSD]2013-05-23 15:42:13.209 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:42:13.209 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136fd0 - 1,2,3 [ CSSD]2013-05-23 15:42:13.209 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c136fd0) pid(14629) version 11:1:1:4 [ CSSD]2013-05-23 15:42:13.225 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:42:13.226 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:42:13.229 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136fd0 [ CSSD]2013-05-23 15:42:13.229 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136fd0) con(0x7f431c12a550) skgpid ospid 14629 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:13.305 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(59/0x7f431c12e6d0) [ CSSD]2013-05-23 15:42:13.305 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 59 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:42:13.305 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c12e6d0 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 15:42:13.305 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:42:13.305 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 740 (0x7f431bfd6210) [ CSSD]2013-05-23 15:42:13.306 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6210 (RPC#740) tag(2e4002c) sent to node 1 [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e4002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014084, gin 1, member 0 [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c091590/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6210 (RPC#740) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6210, ret 0, client 0x7f431c12e6d0 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:13.306 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:42:13.307 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 740 [ CSSD]2013-05-23 15:42:13.315 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 59 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:42:13.315 [1191196992] >TRACE: clssgmExitGrock: client 59 (0x7f431c12e6d0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:42:13.315 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:42:13.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 741 (0x7f431bfd6290) [ CSSD]2013-05-23 15:42:13.315 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6290 (RPC#741) tag(2e5002c) sent to node 1 [ CSSD]2013-05-23 15:42:13.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e5002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:13.315 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:42:13.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6290 (RPC#741) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:13.315 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6290, ret 0, client 0x7f431c12e6d0 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:13.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 741 [ CSSD]2013-05-23 15:42:13.316 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:13.317 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:13.317 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c11ab50) skgpid ospid 14617 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:13.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:13.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:13.779 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:14.655 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:42:14.655 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 742 (0x7f431bfd6310) [ CSSD]2013-05-23 15:42:14.656 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6310 (RPC#742) tag(2e6002c) sent to node 1 [ CSSD]2013-05-23 15:42:14.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:14.656 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 742 (0x7f431bfd6310) [ CSSD]2013-05-23 15:42:14.656 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6310 (RPC#742) state 6, flags 0x202 [ CSSD]2013-05-23 15:42:14.656 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6310, ret 0, client (nil) [ CSSD]2013-05-23 15:42:14.656 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 742 [ CSSD]2013-05-23 15:42:15.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:15.785 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:15.785 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:15.938 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:15.938 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:16.188 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:16.189 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:42:16.190 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:16.190 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c1213e0) pid(14751) version 11:1:1:4 [ CSSD]2013-05-23 15:42:16.195 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:42:16.195 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:16.195 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:42:16.195 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:16.195 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 743 (0x7f431bfd6390) [ CSSD]2013-05-23 15:42:16.195 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6390 (RPC#743) tag(2e7002c) sent to node 1 [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e7002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014085, gin 1, member 0 [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6390 (RPC#743) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6390, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:42:16.196 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 743 [ CSSD]2013-05-23 15:42:16.203 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:16.203 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:16.203 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:16.203 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 744 (0x7f431bfd6410) [ CSSD]2013-05-23 15:42:16.203 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6410 (RPC#744) tag(2e8002c) sent to node 1 [ CSSD]2013-05-23 15:42:16.203 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e8002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:16.203 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:16.203 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6410 (RPC#744) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:16.203 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6410, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:16.204 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 744 [ CSSD]2013-05-23 15:42:16.204 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:16.406 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:16.406 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c12a550) skgpid ospid 14751 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:17.444 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:17.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:17.809 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:19.450 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:19.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:19.842 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:20.941 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:20.941 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:21.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:21.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:21.848 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:23.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:23.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:23.879 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:24.683 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:42:24.683 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 745 (0x7f431bfd6490) [ CSSD]2013-05-23 15:42:24.684 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6490 (RPC#745) tag(2e9002c) sent to node 1 [ CSSD]2013-05-23 15:42:24.684 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:24.684 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 745 (0x7f431bfd6490) [ CSSD]2013-05-23 15:42:24.684 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6490 (RPC#745) state 6, flags 0x202 [ CSSD]2013-05-23 15:42:24.684 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6490, ret 0, client (nil) [ CSSD]2013-05-23 15:42:24.684 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 745 [ CSSD]2013-05-23 15:42:25.493 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:25.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:25.884 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:25.945 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:25.945 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:26.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:26.742 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014086, gin 1, member 0 [ CSSD]2013-05-23 15:42:26.742 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:42:26.742 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:42:26.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:26.750 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:42:26.750 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:42:27.524 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:27.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:27.890 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:29.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:29.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:29.942 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:29.948 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:29.948 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:42:31.438 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:31.439 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:42:31.440 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:31.440 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c1213e0) pid(14890) version 11:1:1:4 [ CSSD]2013-05-23 15:42:31.446 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:42:31.446 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:31.446 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:42:31.446 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:31.446 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 746 (0x7f431bfd6510) [ CSSD]2013-05-23 15:42:31.446 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6510 (RPC#746) tag(2ea002c) sent to node 1 [ CSSD]2013-05-23 15:42:31.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ea002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:31.446 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014087, gin 1, member 0 [ CSSD]2013-05-23 15:42:31.446 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:42:31.447 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:31.447 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:42:31.447 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6510 (RPC#746) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:31.447 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6510, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:31.447 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:42:31.447 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 746 [ CSSD]2013-05-23 15:42:31.453 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:31.453 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:31.453 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:31.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 747 (0x7f431bfd6590) [ CSSD]2013-05-23 15:42:31.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6590 (RPC#747) tag(2eb002c) sent to node 1 [ CSSD]2013-05-23 15:42:31.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02eb002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:31.453 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:31.453 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6590 (RPC#747) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:31.453 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6590, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:31.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 747 [ CSSD]2013-05-23 15:42:31.454 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:31.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:31.656 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:31.656 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c12a550) skgpid ospid 14890 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:31.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:31.948 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:33.547 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:33.960 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:33.960 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:34.707 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:42:34.707 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 748 (0x7f431bfd6610) [ CSSD]2013-05-23 15:42:34.707 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6610 (RPC#748) tag(2ec002c) sent to node 1 [ CSSD]2013-05-23 15:42:34.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:34.708 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 748 (0x7f431bfd6610) [ CSSD]2013-05-23 15:42:34.708 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6610 (RPC#748) state 6, flags 0x202 [ CSSD]2013-05-23 15:42:34.708 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6610, ret 0, client (nil) [ CSSD]2013-05-23 15:42:34.708 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 748 [ CSSD]2013-05-23 15:42:34.951 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:34.951 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:35.552 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:35.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:35.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:37.564 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:37.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:37.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:39.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:39.954 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:39.955 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:39.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:39.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:41.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:41.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:41.993 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014088, gin 1, member 0 [ CSSD]2013-05-23 15:42:41.993 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c120bc0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:42:41.993 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:42:41.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:41.999 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:42:41.999 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129890) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:42:42.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:42.017 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:43.627 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:44.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:44.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:44.731 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:42:44.731 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 749 (0x7f431bfd6690) [ CSSD]2013-05-23 15:42:44.731 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6690 (RPC#749) tag(2ed002c) sent to node 1 [ CSSD]2013-05-23 15:42:44.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:44.732 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 749 (0x7f431bfd6690) [ CSSD]2013-05-23 15:42:44.732 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6690 (RPC#749) state 6, flags 0x202 [ CSSD]2013-05-23 15:42:44.732 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6690, ret 0, client (nil) [ CSSD]2013-05-23 15:42:44.732 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 749 [ CSSD]2013-05-23 15:42:44.958 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:44.958 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:45.633 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:46.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:46.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:46.687 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213e0) allocated [ CSSD]2013-05-23 15:42:46.688 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:42:46.689 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213e0 - 1,2,3 [ CSSD]2013-05-23 15:42:46.689 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c1213e0) pid(15093) version 11:1:1:4 [ CSSD]2013-05-23 15:42:46.695 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1213e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:42:46.695 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:46.695 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 15:42:46.695 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:46.695 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 750 (0x7f431bfd6710) [ CSSD]2013-05-23 15:42:46.696 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6710 (RPC#750) tag(2ee002c) sent to node 1 [ CSSD]2013-05-23 15:42:46.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ee002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:46.696 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014089, gin 1, member 0 [ CSSD]2013-05-23 15:42:46.696 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:42:46.696 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:42:46.696 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:42:46.697 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6710 (RPC#750) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:46.697 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6710, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:46.697 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:42:46.697 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 750 [ CSSD]2013-05-23 15:42:46.703 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:42:46.703 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:46.703 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:42:46.703 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 751 (0x7f431bfd6790) [ CSSD]2013-05-23 15:42:46.703 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6790 (RPC#751) tag(2ef002c) sent to node 1 [ CSSD]2013-05-23 15:42:46.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ef002c) incarnation(174631051) [ CSSD]2013-05-23 15:42:46.704 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:42:46.704 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6790 (RPC#751) state 6, flags 0x100 [ CSSD]2013-05-23 15:42:46.704 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6790, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 15:42:46.704 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 751 [ CSSD]2013-05-23 15:42:46.704 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:42:46.906 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213e0 [ CSSD]2013-05-23 15:42:46.906 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213e0) con(0x7f431c11ab50) skgpid ospid 15093 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:47.645 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:48.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:48.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:48.559 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:42:48.559 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:42:48.559 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:42:48.559 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 15:42:48.559 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 15:42:48.559 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e6720) skgpid 9479 ospid 9479 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:48.577 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:42:48.577 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:42:48.577 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:42:48.577 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 15:42:48.577 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0cf650 [ CSSD]2013-05-23 15:42:48.577 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0cf650) con(0x7f431c0840c0) skgpid 9483 ospid 9483 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:42:49.650 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:49.961 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:49.961 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:50.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:50.094 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:51.676 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:52.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:52.119 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:53.727 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:54.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:54.161 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:54.756 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:42:54.756 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 752 (0x7f431bfd6810) [ CSSD]2013-05-23 15:42:54.756 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6810 (RPC#752) tag(2f0002c) sent to node 1 [ CSSD]2013-05-23 15:42:54.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:54.756 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 752 (0x7f431bfd6810) [ CSSD]2013-05-23 15:42:54.756 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6810 (RPC#752) state 6, flags 0x202 [ CSSD]2013-05-23 15:42:54.756 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6810, ret 0, client (nil) [ CSSD]2013-05-23 15:42:54.757 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 752 [ CSSD]2013-05-23 15:42:54.964 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:54.964 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:42:55.753 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:56.187 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:56.187 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:57.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:57.245 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014090, gin 1, member 0 [ CSSD]2013-05-23 15:42:57.245 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:42:57.245 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:42:57.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:57.251 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:42:57.251 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:42:57.765 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:58.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:42:58.193 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:42:59.771 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:42:59.967 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:42:59.967 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:00.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:00.204 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:01.777 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:01.939 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 15:43:01.940 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:43:01.940 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 15:43:01.940 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1355f0) pid(15256) version 11:1:1:4 [ CSSD]2013-05-23 15:43:01.946 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:43:01.946 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:01.946 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 15:43:01.946 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:01.946 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 753 (0x7f431bfd6890) [ CSSD]2013-05-23 15:43:01.947 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6890 (RPC#753) tag(2f1002c) sent to node 1 [ CSSD]2013-05-23 15:43:01.947 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02f1002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:01.947 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014091, gin 1, member 0 [ CSSD]2013-05-23 15:43:01.947 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:43:01.947 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:43:01.947 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:43:01.948 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6890 (RPC#753) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:01.948 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6890, ret 0, client 0x7f431c121510 member 0x7f431c108da0 [ CSSD]2013-05-23 15:43:01.948 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:43:01.948 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 753 [ CSSD]2013-05-23 15:43:01.954 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:01.954 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:01.954 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:01.954 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 754 (0x7f431bfd6910) [ CSSD]2013-05-23 15:43:01.954 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6910 (RPC#754) tag(2f2002c) sent to node 1 [ CSSD]2013-05-23 15:43:01.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02f2002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:01.955 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:01.955 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6910 (RPC#754) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:01.955 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6910, ret 0, client 0x7f431c121510 member 0x7f431c108da0 [ CSSD]2013-05-23 15:43:01.955 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 754 [ CSSD]2013-05-23 15:43:01.955 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:43:02.157 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 15:43:02.157 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c0e6720) skgpid ospid 15256 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:02.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:02.210 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:03.828 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:04.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:04.262 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:04.783 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:43:04.783 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 755 (0x7f431bfd6990) [ CSSD]2013-05-23 15:43:04.783 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6990 (RPC#755) tag(2f3002c) sent to node 1 [ CSSD]2013-05-23 15:43:04.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:04.784 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 755 (0x7f431bfd6990) [ CSSD]2013-05-23 15:43:04.784 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6990 (RPC#755) state 6, flags 0x202 [ CSSD]2013-05-23 15:43:04.784 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6990, ret 0, client (nil) [ CSSD]2013-05-23 15:43:04.784 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 755 [ CSSD]2013-05-23 15:43:04.970 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:04.970 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:05.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:05.667 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014092, gin 1, member 0 [ CSSD]2013-05-23 15:43:05.667 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:43:05.667 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:43:05.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:05.675 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:43:05.675 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:43:05.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:06.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:06.287 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:07.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:08.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:08.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:08.973 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:08.973 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:43:09.401 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 15:43:09.402 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:43:09.402 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 15:43:09.402 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1355f0) pid(15343) version 11:1:1:4 [ CSSD]2013-05-23 15:43:09.408 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:43:09.408 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:09.408 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:43:09.408 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:09.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 756 (0x7f431bfd6a10) [ CSSD]2013-05-23 15:43:09.409 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6a10 (RPC#756) tag(2f4002c) sent to node 1 [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02f4002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014093, gin 1, member 0 [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6a10 (RPC#756) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6a10, ret 0, client 0x7f431c121510 member 0x7f431c108da0 [ CSSD]2013-05-23 15:43:09.409 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:43:09.410 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 756 [ CSSD]2013-05-23 15:43:09.416 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:09.416 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:09.416 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:09.416 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 757 (0x7f431bfd6a90) [ CSSD]2013-05-23 15:43:09.416 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6a90 (RPC#757) tag(2f5002c) sent to node 1 [ CSSD]2013-05-23 15:43:09.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02f5002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:09.417 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:09.417 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6a90 (RPC#757) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:09.417 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6a90, ret 0, client 0x7f431c121510 member 0x7f431c108da0 [ CSSD]2013-05-23 15:43:09.417 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 757 [ CSSD]2013-05-23 15:43:09.417 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:43:09.619 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 15:43:09.619 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12f0c0) skgpid ospid 15343 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:09.952 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:10.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:10.303 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:11.958 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:12.310 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:12.310 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:12.498 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:12.498 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014094, gin 1, member 0 [ CSSD]2013-05-23 15:43:12.498 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:43:12.498 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:43:12.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:12.504 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:43:12.504 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:43:13.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:13.976 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:13.976 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:14.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:14.362 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:14.812 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:43:14.813 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 758 (0x7f431bfd6b10) [ CSSD]2013-05-23 15:43:14.813 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6b10 (RPC#758) tag(2f6002c) sent to node 1 [ CSSD]2013-05-23 15:43:14.814 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:14.814 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 758 (0x7f431bfd6b10) [ CSSD]2013-05-23 15:43:14.814 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6b10 (RPC#758) state 6, flags 0x202 [ CSSD]2013-05-23 15:43:14.814 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6b10, ret 0, client (nil) [ CSSD]2013-05-23 15:43:14.814 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 758 [ CSSD]2013-05-23 15:43:15.975 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:16.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:16.368 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:17.191 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 15:43:17.192 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:43:17.192 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 15:43:17.192 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1355f0) pid(15406) version 11:1:1:4 [ CSSD]2013-05-23 15:43:17.198 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:43:17.198 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:17.198 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 15:43:17.198 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:17.198 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 759 (0x7f431bfd6b90) [ CSSD]2013-05-23 15:43:17.198 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6b90 (RPC#759) tag(2f7002c) sent to node 1 [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02f7002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014095, gin 1, member 0 [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6b90 (RPC#759) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6b90, ret 0, client 0x7f431c121510 member 0x7f431c108da0 [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:43:17.199 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 759 [ CSSD]2013-05-23 15:43:17.206 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:17.206 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:17.206 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:17.206 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 760 (0x7f431bfd6c10) [ CSSD]2013-05-23 15:43:17.206 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6c10 (RPC#760) tag(2f8002c) sent to node 1 [ CSSD]2013-05-23 15:43:17.206 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02f8002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:17.206 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:17.206 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6c10 (RPC#760) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:17.206 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6c10, ret 0, client 0x7f431c121510 member 0x7f431c108da0 [ CSSD]2013-05-23 15:43:17.206 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 760 [ CSSD]2013-05-23 15:43:17.206 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:43:17.409 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 15:43:17.409 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12f0c0) skgpid ospid 15406 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:18.026 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:18.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:18.393 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:18.979 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:18.979 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:20.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:20.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:20.405 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:22.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:22.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:22.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:23.982 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:23.983 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:24.068 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:24.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:24.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:24.836 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:43:24.837 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 761 (0x7f431bfd6c90) [ CSSD]2013-05-23 15:43:24.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6c90 (RPC#761) tag(2f9002c) sent to node 1 [ CSSD]2013-05-23 15:43:24.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:24.838 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 761 (0x7f431bfd6c90) [ CSSD]2013-05-23 15:43:24.838 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6c90 (RPC#761) state 6, flags 0x202 [ CSSD]2013-05-23 15:43:24.838 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6c90, ret 0, client (nil) [ CSSD]2013-05-23 15:43:24.838 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 761 [ CSSD]2013-05-23 15:43:26.074 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:26.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:26.468 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:27.095 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 15:43:27.096 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:43:27.097 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 15:43:27.097 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c1355f0) pid(15167/15167) version 11:1:1:4 [ CSSD]2013-05-23 15:43:27.114 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:43:27.114 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:43:27.114 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:43:27.114 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 15:43:27.747 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:27.747 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014096, gin 1, member 0 [ CSSD]2013-05-23 15:43:27.747 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:43:27.747 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:43:27.753 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:27.753 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:43:27.753 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:43:27.985 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:27.985 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:43:28.077 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:28.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:28.480 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:30.083 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:30.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:30.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:32.134 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:32.442 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 15:43:32.443 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:43:32.443 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 15:43:32.443 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c13ea10) pid(15544) version 11:1:1:4 [ CSSD]2013-05-23 15:43:32.450 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ea10), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:43:32.450 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:43:32.450 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:43:32.450 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:32.450 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 762 (0x7f431bfd6d10) [ CSSD]2013-05-23 15:43:32.450 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6d10 (RPC#762) tag(2fa002c) sent to node 1 [ CSSD]2013-05-23 15:43:32.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02fa002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:32.450 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014097, gin 1, member 0 [ CSSD]2013-05-23 15:43:32.450 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:43:32.450 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:43:32.450 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:43:32.450 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6d10 (RPC#762) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:32.451 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6d10, ret 0, client 0x7f431c135120 member 0x7f431c123f10 [ CSSD]2013-05-23 15:43:32.451 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:43:32.451 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 762 [ CSSD]2013-05-23 15:43:32.457 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:43:32.457 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:32.457 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c123f10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:32.458 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 763 (0x7f431bfd6d90) [ CSSD]2013-05-23 15:43:32.458 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6d90 (RPC#763) tag(2fb002c) sent to node 1 [ CSSD]2013-05-23 15:43:32.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02fb002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:32.458 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:32.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6d90 (RPC#763) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:32.458 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6d90, ret 0, client 0x7f431c135120 member 0x7f431c123f10 [ CSSD]2013-05-23 15:43:32.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 763 [ CSSD]2013-05-23 15:43:32.458 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c123f10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:43:32.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:32.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:32.661 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 15:43:32.661 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c0cf710) skgpid ospid 15544 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:32.989 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:32.989 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:34.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:34.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:34.563 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:34.868 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:43:34.868 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 764 (0x7f431bfd6e10) [ CSSD]2013-05-23 15:43:34.869 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6e10 (RPC#764) tag(2fc002c) sent to node 1 [ CSSD]2013-05-23 15:43:34.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:34.869 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 764 (0x7f431bfd6e10) [ CSSD]2013-05-23 15:43:34.869 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6e10 (RPC#764) state 6, flags 0x202 [ CSSD]2013-05-23 15:43:34.869 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6e10, ret 0, client (nil) [ CSSD]2013-05-23 15:43:34.869 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 764 [ CSSD]2013-05-23 15:43:35.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:35.956 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014098, gin 1, member 0 [ CSSD]2013-05-23 15:43:35.956 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c04b0c0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 15:43:35.956 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:43:35.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:35.963 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 15:43:35.963 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:43:36.090 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:43:36.090 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:43:36.093 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:43:36.093 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:43:36.095 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:43:36.095 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:43:36.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:36.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:36.589 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:37.992 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:37.992 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:38.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:38.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:38.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:40.183 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:40.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:40.667 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:42.235 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:42.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:42.691 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:42.995 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:42.995 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:43.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:43.001 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014099, gin 1, member 0 [ CSSD]2013-05-23 15:43:43.001 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:43:43.001 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:43:43.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:43.008 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:43:43.008 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:43:44.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:44.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:44.703 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:44.895 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:43:44.895 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 765 (0x7f431bfd6e90) [ CSSD]2013-05-23 15:43:44.895 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6e90 (RPC#765) tag(2fd002c) sent to node 1 [ CSSD]2013-05-23 15:43:44.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:44.896 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 765 (0x7f431bfd6e90) [ CSSD]2013-05-23 15:43:44.896 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6e90 (RPC#765) state 6, flags 0x202 [ CSSD]2013-05-23 15:43:44.896 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6e90, ret 0, client (nil) [ CSSD]2013-05-23 15:43:44.896 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 765 [ CSSD]2013-05-23 15:43:46.266 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:46.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:46.708 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:46.998 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:46.998 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:43:47.692 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 15:43:47.693 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:43:47.694 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 15:43:47.694 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c13ea10) pid(15686) version 11:1:1:4 [ CSSD]2013-05-23 15:43:47.710 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ea10), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:43:47.710 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:43:47.710 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 15:43:47.710 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:47.710 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 766 (0x7f431bfd6f10) [ CSSD]2013-05-23 15:43:47.710 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6f10 (RPC#766) tag(2fe002c) sent to node 1 [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02fe002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014100, gin 1, member 0 [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6f10 (RPC#766) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6f10, ret 0, client 0x7f431c135120 member 0x7f431c123f10 [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:43:47.711 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 766 [ CSSD]2013-05-23 15:43:47.718 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:43:47.718 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:47.718 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c123f10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:43:47.718 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 767 (0x7f431bfd6f90) [ CSSD]2013-05-23 15:43:47.718 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6f90 (RPC#767) tag(2ff002c) sent to node 1 [ CSSD]2013-05-23 15:43:47.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ff002c) incarnation(174631051) [ CSSD]2013-05-23 15:43:47.718 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:43:47.718 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6f90 (RPC#767) state 6, flags 0x100 [ CSSD]2013-05-23 15:43:47.718 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6f90, ret 0, client 0x7f431c135120 member 0x7f431c123f10 [ CSSD]2013-05-23 15:43:47.719 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 767 [ CSSD]2013-05-23 15:43:47.719 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c123f10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:43:47.921 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 15:43:47.921 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c0cf710) skgpid ospid 15686 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:48.278 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:48.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:48.761 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:50.284 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:50.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:50.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:52.001 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:52.002 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:52.336 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:52.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:52.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:52.920 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c10a560) [ CSSD]2013-05-23 15:43:52.920 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c10a560), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:43:52.920 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c10a560), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:43:52.920 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 27 [ CSSD]2013-05-23 15:43:52.929 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:43:52.929 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:43:52.929 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:43:52.929 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 15:43:52.930 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 15:43:52.930 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c109f70) skgpid 10167 ospid 10167 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:52.934 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22ed830) [ CSSD]2013-05-23 15:43:52.935 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22ed830), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:43:52.935 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22ed830), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:43:52.935 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 15:43:52.938 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23029f0) [ CSSD]2013-05-23 15:43:52.938 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23029f0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:43:52.938 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23029f0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:43:52.938 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 15:43:52.939 [1191196992] >TRACE: clssgmDeadProc: proc 0x23024c0 [ CSSD]2013-05-23 15:43:52.939 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23024c0) con(0x22f84f0) skgpid 10165 ospid 10165 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:43:54.341 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:54.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:54.784 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:54.920 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:43:54.920 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 768 (0x7f431bfd7010) [ CSSD]2013-05-23 15:43:54.920 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7010 (RPC#768) tag(300002c) sent to node 1 [ CSSD]2013-05-23 15:43:54.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:54.921 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 768 (0x7f431bfd7010) [ CSSD]2013-05-23 15:43:54.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7010 (RPC#768) state 6, flags 0x202 [ CSSD]2013-05-23 15:43:54.921 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7010, ret 0, client (nil) [ CSSD]2013-05-23 15:43:54.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 768 [ CSSD]2013-05-23 15:43:56.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:56.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:56.790 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:43:57.005 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:43:57.005 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:43:58.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:58.252 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014101, gin 1, member 0 [ CSSD]2013-05-23 15:43:58.252 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:43:58.252 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:43:58.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:58.259 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:43:58.259 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:43:58.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:43:58.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:43:58.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:00.381 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:00.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:00.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:02.008 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:02.008 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:02.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:02.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:02.823 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:02.953 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 15:44:02.954 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:44:02.955 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 15:44:02.955 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c10a190) pid(15850) version 11:1:1:4 [ CSSD]2013-05-23 15:44:02.971 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a190), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:44:02.971 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:44:02.971 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 15:44:02.971 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:02.971 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 769 (0x7f431bfd7090) [ CSSD]2013-05-23 15:44:02.971 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7090 (RPC#769) tag(301002c) sent to node 1 [ CSSD]2013-05-23 15:44:02.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0301002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:02.971 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014102, gin 1, member 0 [ CSSD]2013-05-23 15:44:02.971 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:44:02.972 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:44:02.972 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:44:02.972 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7090 (RPC#769) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:02.972 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7090, ret 0, client 0x7f431c135120 member 0x7f431c13f950 [ CSSD]2013-05-23 15:44:02.972 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:44:02.972 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 769 [ CSSD]2013-05-23 15:44:02.979 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:44:02.979 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:02.979 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:02.979 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 770 (0x7f431bfd7110) [ CSSD]2013-05-23 15:44:02.979 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7110 (RPC#770) tag(302002c) sent to node 1 [ CSSD]2013-05-23 15:44:02.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0302002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:02.979 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:02.979 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7110 (RPC#770) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:02.979 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7110, ret 0, client 0x7f431c135120 member 0x7f431c13f950 [ CSSD]2013-05-23 15:44:02.980 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 770 [ CSSD]2013-05-23 15:44:02.980 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:44:03.182 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 15:44:03.182 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c109f70) skgpid ospid 15850 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:44:04.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:04.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:04.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:04.953 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:44:04.953 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 771 (0x7f431bfd7190) [ CSSD]2013-05-23 15:44:04.954 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7190 (RPC#771) tag(303002c) sent to node 1 [ CSSD]2013-05-23 15:44:04.954 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:04.954 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 771 (0x7f431bfd7190) [ CSSD]2013-05-23 15:44:04.954 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7190 (RPC#771) state 6, flags 0x202 [ CSSD]2013-05-23 15:44:04.954 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7190, ret 0, client (nil) [ CSSD]2013-05-23 15:44:04.954 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 771 [ CSSD]2013-05-23 15:44:05.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:05.925 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014103, gin 1, member 0 [ CSSD]2013-05-23 15:44:05.925 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:44:05.925 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:44:05.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:05.932 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:44:05.932 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:44:06.011 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:06.011 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:44:06.448 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:06.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:06.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:07.767 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 15:44:07.768 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:44:07.769 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 15:44:07.769 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c10a190) pid(15159/15159) version 11:1:1:4 [ CSSD]2013-05-23 15:44:07.785 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a190), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 15:44:07.785 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:44:07.785 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:44:07.785 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 15:44:08.459 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:08.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:08.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:08.958 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(2/0x7f431c12e6d0) [ CSSD]2013-05-23 15:44:08.958 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:44:08.958 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:44:08.958 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 15:44:09.664 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 15:44:09.665 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:44:09.666 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 15:44:09.666 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c12b920) pid(15935) version 11:1:1:4 [ CSSD]2013-05-23 15:44:09.672 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c108e70) [ CSSD]2013-05-23 15:44:09.672 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:44:09.672 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:44:09.672 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:09.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 772 (0x7f431bfd7210) [ CSSD]2013-05-23 15:44:09.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7210 (RPC#772) tag(304002c) sent to node 1 [ CSSD]2013-05-23 15:44:09.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0304002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:09.672 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014104, gin 1, member 0 [ CSSD]2013-05-23 15:44:09.672 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:44:09.673 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:44:09.673 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:44:09.673 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7210 (RPC#772) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:09.673 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7210, ret 0, client 0x7f431c108e70 member 0x7f431c0958d0 [ CSSD]2013-05-23 15:44:09.673 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 15:44:09.673 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 772 [ CSSD]2013-05-23 15:44:09.680 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:44:09.680 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:09.680 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0958d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:09.680 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 773 (0x7f431bfd7290) [ CSSD]2013-05-23 15:44:09.680 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7290 (RPC#773) tag(305002c) sent to node 1 [ CSSD]2013-05-23 15:44:09.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0305002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:09.680 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:09.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7290 (RPC#773) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:09.680 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7290, ret 0, client 0x7f431c108e70 member 0x7f431c0958d0 [ CSSD]2013-05-23 15:44:09.680 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 773 [ CSSD]2013-05-23 15:44:09.680 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0958d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:44:09.883 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 15:44:09.883 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c12f0c0) skgpid ospid 15935 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:44:10.464 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:10.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:10.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:11.014 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:11.014 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:12.477 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:12.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:12.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:13.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:13.504 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014105, gin 1, member 0 [ CSSD]2013-05-23 15:44:13.504 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:44:13.504 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:44:13.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:13.510 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:44:13.510 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e55a0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:44:14.482 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:14.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:14.925 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:14.983 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:44:14.983 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 774 (0x7f431bfd7310) [ CSSD]2013-05-23 15:44:14.983 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7310 (RPC#774) tag(306002c) sent to node 1 [ CSSD]2013-05-23 15:44:14.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:14.984 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 774 (0x7f431bfd7310) [ CSSD]2013-05-23 15:44:14.984 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7310 (RPC#774) state 6, flags 0x202 [ CSSD]2013-05-23 15:44:14.984 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7310, ret 0, client (nil) [ CSSD]2013-05-23 15:44:14.984 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 774 [ CSSD]2013-05-23 15:44:16.017 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:16.017 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:16.488 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:16.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:16.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:18.215 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b850) allocated [ CSSD]2013-05-23 15:44:18.216 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:44:18.217 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b850 - 1,2,3 [ CSSD]2013-05-23 15:44:18.217 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c12b850) pid(16026) version 11:1:1:4 [ CSSD]2013-05-23 15:44:18.223 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b850), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:44:18.223 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:44:18.223 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 15:44:18.223 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:18.223 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 775 (0x7f431bfd7390) [ CSSD]2013-05-23 15:44:18.223 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7390 (RPC#775) tag(307002c) sent to node 1 [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0307002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014106, gin 1, member 0 [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7390 (RPC#775) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7390, ret 0, client 0x7f431c135120 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:44:18.224 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 775 [ CSSD]2013-05-23 15:44:18.231 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:44:18.231 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:18.231 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0959a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:18.231 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 776 (0x7f431bfd7410) [ CSSD]2013-05-23 15:44:18.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7410 (RPC#776) tag(308002c) sent to node 1 [ CSSD]2013-05-23 15:44:18.231 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0308002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:18.231 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:18.231 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7410 (RPC#776) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:18.231 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7410, ret 0, client 0x7f431c135120 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:44:18.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 776 [ CSSD]2013-05-23 15:44:18.232 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0959a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:44:18.434 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b850 [ CSSD]2013-05-23 15:44:18.434 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b850) con(0x7f431c0e5900) skgpid ospid 16026 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:44:18.560 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:18.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:18.983 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:20.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:21.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:21.008 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:21.020 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:21.020 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:22.637 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:22.906 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:44:22.906 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:44:22.907 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:44:22.908 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:44:22.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:22.989 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014107, gin 1, member 0 [ CSSD]2013-05-23 15:44:22.989 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c1215d0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 15:44:22.989 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 15:44:22.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:22.995 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 15:44:22.995 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c0e55a0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:44:23.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:23.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:24.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:25.010 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:44:25.010 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 777 (0x7f431bfd7490) [ CSSD]2013-05-23 15:44:25.010 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7490 (RPC#777) tag(309002c) sent to node 1 [ CSSD]2013-05-23 15:44:25.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:25.011 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 777 (0x7f431bfd7490) [ CSSD]2013-05-23 15:44:25.011 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7490 (RPC#777) state 6, flags 0x202 [ CSSD]2013-05-23 15:44:25.011 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7490, ret 0, client (nil) [ CSSD]2013-05-23 15:44:25.011 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 777 [ CSSD]2013-05-23 15:44:25.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:25.025 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:26.023 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:26.023 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:26.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:27.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:27.030 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:28.660 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:28.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:28.756 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014108, gin 1, member 0 [ CSSD]2013-05-23 15:44:28.756 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:44:28.756 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:44:28.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:28.763 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:44:28.763 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e55a0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:44:29.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:29.083 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:30.026 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:30.026 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:44:30.685 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:31.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:31.088 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:32.738 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:33.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:33.094 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:33.466 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 15:44:33.466 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:44:33.467 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 15:44:33.467 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c12b920) pid(16187) version 11:1:1:4 [ CSSD]2013-05-23 15:44:33.472 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:44:33.472 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:44:33.472 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 15:44:33.472 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:33.472 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 778 (0x7f431bfd7510) [ CSSD]2013-05-23 15:44:33.473 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7510 (RPC#778) tag(30a002c) sent to node 1 [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(030a002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014109, gin 1, member 0 [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7510 (RPC#778) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7510, ret 0, client 0x7f431c10a560 member 0x7f431c13edd0 [ CSSD]2013-05-23 15:44:33.473 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:44:33.474 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 778 [ CSSD]2013-05-23 15:44:33.480 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:44:33.480 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:33.480 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:33.480 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 779 (0x7f431bfd7590) [ CSSD]2013-05-23 15:44:33.480 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7590 (RPC#779) tag(30b002c) sent to node 1 [ CSSD]2013-05-23 15:44:33.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(030b002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:33.480 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:33.480 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7590 (RPC#779) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:33.480 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7590, ret 0, client 0x7f431c10a560 member 0x7f431c13edd0 [ CSSD]2013-05-23 15:44:33.481 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 779 [ CSSD]2013-05-23 15:44:33.481 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:44:33.683 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 15:44:33.683 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c0e6720) skgpid ospid 16187 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:44:34.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:35.029 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:35.029 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:35.032 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:44:35.033 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 780 (0x7f431bfd7610) [ CSSD]2013-05-23 15:44:35.033 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7610 (RPC#780) tag(30c002c) sent to node 1 [ CSSD]2013-05-23 15:44:35.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:35.034 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 780 (0x7f431bfd7610) [ CSSD]2013-05-23 15:44:35.034 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7610 (RPC#780) state 6, flags 0x202 [ CSSD]2013-05-23 15:44:35.034 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7610, ret 0, client (nil) [ CSSD]2013-05-23 15:44:35.034 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 780 [ CSSD]2013-05-23 15:44:35.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:35.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:36.748 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:37.131 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:37.131 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:38.760 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:39.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:39.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:40.032 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:40.032 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:40.785 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:41.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:41.209 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:42.858 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:43.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:43.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:44.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:44.005 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014110, gin 1, member 0 [ CSSD]2013-05-23 15:44:44.005 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:44:44.005 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:44:44.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:44.012 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:44:44.012 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:44:44.864 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:45.035 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:45.035 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:45.057 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:44:45.058 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 781 (0x7f431bfd7690) [ CSSD]2013-05-23 15:44:45.058 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7690 (RPC#781) tag(30d002c) sent to node 1 [ CSSD]2013-05-23 15:44:45.058 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:45.058 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 781 (0x7f431bfd7690) [ CSSD]2013-05-23 15:44:45.058 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7690 (RPC#781) state 6, flags 0x202 [ CSSD]2013-05-23 15:44:45.058 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7690, ret 0, client (nil) [ CSSD]2013-05-23 15:44:45.059 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 781 [ CSSD]2013-05-23 15:44:45.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:45.245 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:46.870 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:47.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:47.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:48.715 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 15:44:48.716 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:44:48.717 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 15:44:48.717 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c0958d0) pid(16364) version 11:1:1:4 [ CSSD]2013-05-23 15:44:48.722 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0958d0), client(1/0x7f431c108e70) [ CSSD]2013-05-23 15:44:48.722 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:44:48.722 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 15:44:48.723 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:48.723 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 782 (0x7f431bfd7710) [ CSSD]2013-05-23 15:44:48.723 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7710 (RPC#782) tag(30e002c) sent to node 1 [ CSSD]2013-05-23 15:44:48.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(030e002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:48.723 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014111, gin 1, member 0 [ CSSD]2013-05-23 15:44:48.723 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:44:48.724 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:44:48.724 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:44:48.724 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7710 (RPC#782) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:48.724 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7710, ret 0, client 0x7f431c108e70 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:44:48.724 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 15:44:48.724 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 782 [ CSSD]2013-05-23 15:44:48.731 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:44:48.731 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:48.731 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:44:48.731 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 783 (0x7f431bfd7790) [ CSSD]2013-05-23 15:44:48.731 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7790 (RPC#783) tag(30f002c) sent to node 1 [ CSSD]2013-05-23 15:44:48.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(030f002c) incarnation(174631051) [ CSSD]2013-05-23 15:44:48.731 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:44:48.731 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7790 (RPC#783) state 6, flags 0x100 [ CSSD]2013-05-23 15:44:48.731 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7790, ret 0, client 0x7f431c108e70 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:44:48.731 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 783 [ CSSD]2013-05-23 15:44:48.731 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:44:48.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:48.934 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 15:44:48.934 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c0e6720) skgpid ospid 16364 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:44:49.037 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:49.037 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:44:49.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:49.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:50.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:51.290 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:51.290 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:52.949 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:53.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:53.314 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:54.041 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:54.041 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:54.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:55.080 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:44:55.080 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 784 (0x7f431bfd7810) [ CSSD]2013-05-23 15:44:55.081 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7810 (RPC#784) tag(310002c) sent to node 1 [ CSSD]2013-05-23 15:44:55.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:55.081 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 784 (0x7f431bfd7810) [ CSSD]2013-05-23 15:44:55.081 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7810 (RPC#784) state 6, flags 0x202 [ CSSD]2013-05-23 15:44:55.081 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7810, ret 0, client (nil) [ CSSD]2013-05-23 15:44:55.081 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 784 [ CSSD]2013-05-23 15:44:55.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:55.325 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:56.961 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:57.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:57.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:44:58.972 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:44:59.044 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:44:59.044 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:44:59.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:59.276 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014112, gin 1, member 0 [ CSSD]2013-05-23 15:44:59.276 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:44:59.276 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:44:59.283 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:59.283 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:44:59.283 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:44:59.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:44:59.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:00.977 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:01.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:01.390 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:02.988 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:03.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:03.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:03.975 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1215d0) allocated [ CSSD]2013-05-23 15:45:03.976 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:45:03.976 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1215d0 - 1,2,3 [ CSSD]2013-05-23 15:45:03.976 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c1215d0) pid(16581) version 11:1:1:4 [ CSSD]2013-05-23 15:45:03.982 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1215d0), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:45:03.982 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:45:03.982 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 15:45:03.982 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:03.982 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 785 (0x7f431bfd7890) [ CSSD]2013-05-23 15:45:03.982 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7890 (RPC#785) tag(311002c) sent to node 1 [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0311002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014113, gin 1, member 0 [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7890 (RPC#785) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7890, ret 0, client 0x7f431c135120 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:45:03.983 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 785 [ CSSD]2013-05-23 15:45:03.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:45:03.989 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:03.989 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:03.989 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 786 (0x7f431bfd7910) [ CSSD]2013-05-23 15:45:03.989 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7910 (RPC#786) tag(312002c) sent to node 1 [ CSSD]2013-05-23 15:45:03.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0312002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:03.989 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:03.989 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7910 (RPC#786) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:03.989 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7910, ret 0, client 0x7f431c135120 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:45:03.990 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 786 [ CSSD]2013-05-23 15:45:03.990 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:45:04.047 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:04.048 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:04.192 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1215d0 [ CSSD]2013-05-23 15:45:04.192 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1215d0) con(0x7f431c0e5900) skgpid ospid 16581 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:04.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:05.105 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:45:05.105 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 787 (0x7f431bfd7990) [ CSSD]2013-05-23 15:45:05.105 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7990 (RPC#787) tag(313002c) sent to node 1 [ CSSD]2013-05-23 15:45:05.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:05.106 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 787 (0x7f431bfd7990) [ CSSD]2013-05-23 15:45:05.106 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7990 (RPC#787) state 6, flags 0x202 [ CSSD]2013-05-23 15:45:05.106 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7990, ret 0, client (nil) [ CSSD]2013-05-23 15:45:05.106 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 787 [ CSSD]2013-05-23 15:45:05.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:05.406 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:06.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:06.185 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014114, gin 1, member 0 [ CSSD]2013-05-23 15:45:06.185 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:45:06.185 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:45:06.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:06.191 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:45:06.192 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:45:07.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:07.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:07.411 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:07.589 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 15:45:07.590 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:45:07.591 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 15:45:07.591 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0958d0) pid(16658) version 11:1:1:4 [ CSSD]2013-05-23 15:45:08.050 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:08.050 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:45:08.202 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 15:45:08.202 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c12f0c0) skgpid ospid 16658 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:09.052 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:09.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:09.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:09.924 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1215d0) allocated [ CSSD]2013-05-23 15:45:09.925 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:45:09.925 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1215d0 - 1,2,3 [ CSSD]2013-05-23 15:45:09.925 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c1215d0) pid(16772) version 11:1:1:4 [ CSSD]2013-05-23 15:45:09.931 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1215d0), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:45:09.931 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:45:09.931 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:45:09.931 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:09.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 788 (0x7f431bfd7a10) [ CSSD]2013-05-23 15:45:09.932 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7a10 (RPC#788) tag(314002c) sent to node 1 [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0314002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014115, gin 1, member 0 [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c123c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7a10 (RPC#788) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7a10, ret 0, client 0x7f431c135120 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:45:09.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:45:09.933 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 788 [ CSSD]2013-05-23 15:45:09.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:45:09.939 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:09.939 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:09.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 789 (0x7f431bfd7a90) [ CSSD]2013-05-23 15:45:09.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7a90 (RPC#789) tag(315002c) sent to node 1 [ CSSD]2013-05-23 15:45:09.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0315002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:09.939 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:09.939 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7a90 (RPC#789) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:09.939 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7a90, ret 0, client 0x7f431c135120 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:45:09.940 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 789 [ CSSD]2013-05-23 15:45:09.940 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:45:10.142 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1215d0 [ CSSD]2013-05-23 15:45:10.142 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1215d0) con(0x7f431c0cf710) skgpid ospid 16772 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:11.118 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:11.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:11.511 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:13.053 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:13.054 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:13.150 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:13.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:13.522 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:14.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:14.527 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014116, gin 1, member 0 [ CSSD]2013-05-23 15:45:14.527 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:45:14.527 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:45:14.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:14.535 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:45:14.535 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:45:15.132 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:45:15.133 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 790 (0x7f431bfd7b10) [ CSSD]2013-05-23 15:45:15.133 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7b10 (RPC#790) tag(316002c) sent to node 1 [ CSSD]2013-05-23 15:45:15.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:15.133 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 790 (0x7f431bfd7b10) [ CSSD]2013-05-23 15:45:15.133 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7b10 (RPC#790) state 6, flags 0x202 [ CSSD]2013-05-23 15:45:15.133 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7b10, ret 0, client (nil) [ CSSD]2013-05-23 15:45:15.133 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 790 [ CSSD]2013-05-23 15:45:15.155 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:15.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:15.528 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:17.168 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:17.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:17.534 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:18.057 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:18.057 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:18.599 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2312950) [ CSSD]2013-05-23 15:45:18.599 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2312950), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:45:18.599 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2312950), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:45:18.599 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 15:45:18.602 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22ed3f0) [ CSSD]2013-05-23 15:45:18.602 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22ed3f0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:45:18.602 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22ed3f0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:45:18.602 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 15:45:18.603 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2302f30) [ CSSD]2013-05-23 15:45:18.603 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2302f30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:45:18.603 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2302f30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:45:18.603 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 15:45:18.603 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129a90 [ CSSD]2013-05-23 15:45:18.603 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129a90) con(0x7f431c108910) skgpid 11345 ospid 11345 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:18.605 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e2fe0) [ CSSD]2013-05-23 15:45:18.605 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e2fe0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:45:18.605 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e2fe0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:45:18.605 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 15:45:18.606 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e2eb0 [ CSSD]2013-05-23 15:45:18.606 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e2eb0) con(0x22fdb60) skgpid 11343 ospid 11343 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:19.173 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:19.224 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:45:19.225 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:45:19.225 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:45:19.225 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c0955a0) pid(16856) version 11:1:1:4 [ CSSD]2013-05-23 15:45:19.230 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:45:19.230 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:45:19.230 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:45:19.230 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:19.230 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 791 (0x7f431bfd7b90) [ CSSD]2013-05-23 15:45:19.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7b90 (RPC#791) tag(317002c) sent to node 1 [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0317002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014117, gin 1, member 0 [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7b90 (RPC#791) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7b90, ret 0, client 0x7f431c10a560 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:45:19.231 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:45:19.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 791 [ CSSD]2013-05-23 15:45:19.238 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:45:19.238 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:19.238 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0959a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:19.238 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 792 (0x7f431bfd7c10) [ CSSD]2013-05-23 15:45:19.238 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7c10 (RPC#792) tag(318002c) sent to node 1 [ CSSD]2013-05-23 15:45:19.238 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0318002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:19.238 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:19.238 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7c10 (RPC#792) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:19.238 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7c10, ret 0, client 0x7f431c10a560 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:45:19.238 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 792 [ CSSD]2013-05-23 15:45:19.239 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0959a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:45:19.441 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:45:19.441 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c0e5900) skgpid ospid 16856 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:19.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:19.555 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:21.198 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:21.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:21.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:23.060 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:23.060 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:23.250 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:23.593 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:23.593 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:25.158 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:45:25.159 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 793 (0x7f431bfd7c90) [ CSSD]2013-05-23 15:45:25.159 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7c90 (RPC#793) tag(319002c) sent to node 1 [ CSSD]2013-05-23 15:45:25.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:25.159 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 793 (0x7f431bfd7c90) [ CSSD]2013-05-23 15:45:25.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7c90 (RPC#793) state 6, flags 0x202 [ CSSD]2013-05-23 15:45:25.159 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7c90, ret 0, client (nil) [ CSSD]2013-05-23 15:45:25.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 793 [ CSSD]2013-05-23 15:45:25.256 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:25.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:25.598 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:27.063 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:27.063 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:45:27.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:27.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:27.623 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:29.274 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:29.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:29.635 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:29.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:29.793 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014118, gin 1, member 0 [ CSSD]2013-05-23 15:45:29.793 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:45:29.793 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:45:29.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:29.800 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:45:29.800 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:45:31.280 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:31.641 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:31.641 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:32.066 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:32.066 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:33.291 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:33.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:33.693 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:34.473 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:45:34.474 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:45:34.475 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:45:34.475 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c0955a0) pid(16997) version 11:1:1:4 [ CSSD]2013-05-23 15:45:34.480 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c108e70) [ CSSD]2013-05-23 15:45:34.480 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:45:34.480 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 15:45:34.480 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:34.480 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 794 (0x7f431bfd7d10) [ CSSD]2013-05-23 15:45:34.480 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7d10 (RPC#794) tag(31a002c) sent to node 1 [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(031a002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014119, gin 1, member 0 [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7d10 (RPC#794) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7d10, ret 0, client 0x7f431c108e70 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 15:45:34.481 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 794 [ CSSD]2013-05-23 15:45:34.488 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:45:34.488 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:34.488 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:34.488 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 795 (0x7f431bfd7d90) [ CSSD]2013-05-23 15:45:34.488 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7d90 (RPC#795) tag(31b002c) sent to node 1 [ CSSD]2013-05-23 15:45:34.488 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(031b002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:34.488 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:34.488 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7d90 (RPC#795) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:34.488 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7d90, ret 0, client 0x7f431c108e70 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:45:34.489 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 795 [ CSSD]2013-05-23 15:45:34.489 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:45:34.691 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:45:34.691 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c0e5900) skgpid ospid 16997 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:35.182 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:45:35.182 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 796 (0x7f431bfd7e10) [ CSSD]2013-05-23 15:45:35.182 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7e10 (RPC#796) tag(31c002c) sent to node 1 [ CSSD]2013-05-23 15:45:35.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:35.183 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 796 (0x7f431bfd7e10) [ CSSD]2013-05-23 15:45:35.183 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7e10 (RPC#796) state 6, flags 0x202 [ CSSD]2013-05-23 15:45:35.183 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7e10, ret 0, client (nil) [ CSSD]2013-05-23 15:45:35.183 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 796 [ CSSD]2013-05-23 15:45:35.296 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:35.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:35.699 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:37.070 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:37.070 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:37.348 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:37.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:37.724 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:39.353 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:39.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:39.736 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:41.358 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:41.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:41.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:42.073 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:42.073 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:43.451 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:43.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:43.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:45.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:45.064 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014120, gin 1, member 0 [ CSSD]2013-05-23 15:45:45.064 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:45:45.064 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:45:45.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:45.072 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:45:45.072 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f280) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:45:45.206 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:45:45.207 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 797 (0x7f431bfd7e90) [ CSSD]2013-05-23 15:45:45.207 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7e90 (RPC#797) tag(31d002c) sent to node 1 [ CSSD]2013-05-23 15:45:45.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:45.207 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 797 (0x7f431bfd7e90) [ CSSD]2013-05-23 15:45:45.207 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7e90 (RPC#797) state 6, flags 0x202 [ CSSD]2013-05-23 15:45:45.207 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7e90, ret 0, client (nil) [ CSSD]2013-05-23 15:45:45.207 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 797 [ CSSD]2013-05-23 15:45:45.476 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:45.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:45.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:47.076 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:47.076 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:47.488 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:47.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:47.805 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:49.494 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:49.734 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:45:49.735 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:45:49.736 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:45:49.736 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0955a0) pid(17162) version 11:1:1:4 [ CSSD]2013-05-23 15:45:49.742 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:45:49.742 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:45:49.742 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:45:49.742 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:49.742 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 798 (0x7f431bfd7f10) [ CSSD]2013-05-23 15:45:49.742 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7f10 (RPC#798) tag(31e002c) sent to node 1 [ CSSD]2013-05-23 15:45:49.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(031e002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:49.742 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014121, gin 1, member 0 [ CSSD]2013-05-23 15:45:49.742 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:45:49.743 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:45:49.743 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:45:49.743 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7f10 (RPC#798) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:49.743 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7f10, ret 0, client 0x7f431c135120 member 0x7f431c1216a0 [ CSSD]2013-05-23 15:45:49.743 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:45:49.743 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 798 [ CSSD]2013-05-23 15:45:49.749 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:45:49.749 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:49.749 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1216a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:45:49.749 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 799 (0x7f431bfd7f90) [ CSSD]2013-05-23 15:45:49.749 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7f90 (RPC#799) tag(31f002c) sent to node 1 [ CSSD]2013-05-23 15:45:49.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(031f002c) incarnation(174631051) [ CSSD]2013-05-23 15:45:49.750 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:45:49.750 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7f90 (RPC#799) state 6, flags 0x100 [ CSSD]2013-05-23 15:45:49.750 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7f90, ret 0, client 0x7f431c135120 member 0x7f431c1216a0 [ CSSD]2013-05-23 15:45:49.750 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 799 [ CSSD]2013-05-23 15:45:49.750 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1216a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:45:49.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:49.816 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:49.952 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:45:49.952 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c12f0c0) skgpid ospid 17162 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:45:51.500 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:51.822 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:51.822 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:52.079 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:52.080 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:53.551 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:53.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:53.913 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:55.236 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:45:55.236 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 800 (0x7f431bfd8010) [ CSSD]2013-05-23 15:45:55.237 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8010 (RPC#800) tag(320002c) sent to node 1 [ CSSD]2013-05-23 15:45:55.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:55.237 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 800 (0x7f431bfd8010) [ CSSD]2013-05-23 15:45:55.237 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8010 (RPC#800) state 6, flags 0x202 [ CSSD]2013-05-23 15:45:55.237 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8010, ret 0, client (nil) [ CSSD]2013-05-23 15:45:55.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 800 [ CSSD]2013-05-23 15:45:55.574 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:55.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:55.938 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:57.082 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:45:57.082 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:45:57.588 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:57.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:57.943 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:45:59.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:45:59.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:45:59.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:00.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:00.323 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014122, gin 1, member 0 [ CSSD]2013-05-23 15:46:00.323 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:46:00.323 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:46:00.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:00.329 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:46:00.329 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:46:01.600 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:02.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:02.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:02.085 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:02.085 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:03.620 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:04.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:04.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:04.984 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:46:04.985 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:46:04.985 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:46:04.986 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0955a0) pid(17314) version 11:1:1:4 [ CSSD]2013-05-23 15:46:04.991 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:46:04.991 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:46:04.991 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 15:46:04.991 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:04.992 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 801 (0x7f431bfd8090) [ CSSD]2013-05-23 15:46:04.992 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8090 (RPC#801) tag(321002c) sent to node 1 [ CSSD]2013-05-23 15:46:04.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0321002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:04.992 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014123, gin 1, member 0 [ CSSD]2013-05-23 15:46:04.992 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:46:04.993 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:46:04.993 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:46:04.993 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8090 (RPC#801) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:04.993 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8090, ret 0, client 0x7f431c10a560 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:46:04.993 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:46:04.993 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 801 [ CSSD]2013-05-23 15:46:04.999 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:46:04.999 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:04.999 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:04.999 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 802 (0x7f431bfd8110) [ CSSD]2013-05-23 15:46:04.999 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8110 (RPC#802) tag(322002c) sent to node 1 [ CSSD]2013-05-23 15:46:05.000 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0322002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:05.000 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:05.000 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8110 (RPC#802) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:05.000 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8110, ret 0, client 0x7f431c10a560 member 0x7f431c13eea0 [ CSSD]2013-05-23 15:46:05.000 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 802 [ CSSD]2013-05-23 15:46:05.000 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:46:05.203 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:46:05.203 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c12f0c0) skgpid ospid 17314 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:05.273 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:46:05.273 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 803 (0x7f431bfd8190) [ CSSD]2013-05-23 15:46:05.273 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8190 (RPC#803) tag(323002c) sent to node 1 [ CSSD]2013-05-23 15:46:05.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:05.274 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 803 (0x7f431bfd8190) [ CSSD]2013-05-23 15:46:05.274 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8190 (RPC#803) state 6, flags 0x202 [ CSSD]2013-05-23 15:46:05.274 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8190, ret 0, client (nil) [ CSSD]2013-05-23 15:46:05.274 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 803 [ CSSD]2013-05-23 15:46:05.625 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:06.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:06.060 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:06.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:06.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014124, gin 1, member 0 [ CSSD]2013-05-23 15:46:06.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:46:06.442 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:46:06.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:06.449 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:46:06.449 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f280) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:46:07.088 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:07.089 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:07.660 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:08.065 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:08.065 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:09.665 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:10.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:10.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:10.192 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:46:10.193 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:46:10.194 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:46:10.194 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0955a0) pid(17395) version 11:1:1:4 [ CSSD]2013-05-23 15:46:10.200 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c108e70) [ CSSD]2013-05-23 15:46:10.200 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:46:10.200 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0e5900, requested num -1 [ CSSD]2013-05-23 15:46:10.200 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:10.200 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 804 (0x7f431bfd8210) [ CSSD]2013-05-23 15:46:10.200 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8210 (RPC#804) tag(324002c) sent to node 1 [ CSSD]2013-05-23 15:46:10.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0324002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:10.200 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014125, gin 1, member 0 [ CSSD]2013-05-23 15:46:10.200 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:46:10.201 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:46:10.201 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:46:10.201 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8210 (RPC#804) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:10.201 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8210, ret 0, client 0x7f431c108e70 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:46:10.201 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 15:46:10.201 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 804 [ CSSD]2013-05-23 15:46:10.207 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:46:10.207 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:10.207 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0959a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:10.207 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 805 (0x7f431bfd8290) [ CSSD]2013-05-23 15:46:10.207 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8290 (RPC#805) tag(325002c) sent to node 1 [ CSSD]2013-05-23 15:46:10.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0325002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:10.207 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:10.208 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8290 (RPC#805) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:10.208 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8290, ret 0, client 0x7f431c108e70 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:46:10.208 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 805 [ CSSD]2013-05-23 15:46:10.208 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0959a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:46:10.410 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:46:10.410 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c12f0c0) skgpid ospid 17395 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:11.091 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:11.091 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:46:11.730 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:12.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:12.104 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:13.783 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:14.115 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:14.115 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:15.092 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:46:15.093 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:46:15.093 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:46:15.093 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c0955a0) pid(17435) version 11:1:1:4 [ CSSD]2013-05-23 15:46:15.099 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:46:15.099 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:46:15.099 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c135120 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 15:46:15.099 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:46:15.099 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 806 (0x7f431bfd8310) [ CSSD]2013-05-23 15:46:15.099 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8310 (RPC#806) tag(326002c) sent to node 1 [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0326002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014126, gin 1, member 0 [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8310 (RPC#806) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8310, ret 0, client 0x7f431c135120 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:46:15.100 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 806 [ CSSD]2013-05-23 15:46:15.106 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:46:15.106 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:46:15.106 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0959a0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:46:15.106 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 807 (0x7f431bfd8390) [ CSSD]2013-05-23 15:46:15.106 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8390 (RPC#807) tag(327002c) sent to node 1 [ CSSD]2013-05-23 15:46:15.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0327002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:15.106 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:46:15.106 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8390 (RPC#807) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:15.106 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8390, ret 0, client 0x7f431c135120 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:46:15.107 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 807 [ CSSD]2013-05-23 15:46:15.107 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c0959a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:46:15.219 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c124850) allocated [ CSSD]2013-05-23 15:46:15.220 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:46:15.220 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c124850 - 1,2,3 [ CSSD]2013-05-23 15:46:15.220 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c124850) pid(17440) version 11:1:1:4 [ CSSD]2013-05-23 15:46:15.238 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:46:15.239 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:46:15.242 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c124850 [ CSSD]2013-05-23 15:46:15.242 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c124850) con(0x7f431c0e6720) skgpid ospid 17440 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:15.300 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:46:15.300 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 808 (0x7f431bfd8410) [ CSSD]2013-05-23 15:46:15.300 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8410 (RPC#808) tag(328002c) sent to node 1 [ CSSD]2013-05-23 15:46:15.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:15.301 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 808 (0x7f431bfd8410) [ CSSD]2013-05-23 15:46:15.301 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8410 (RPC#808) state 6, flags 0x202 [ CSSD]2013-05-23 15:46:15.301 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8410, ret 0, client (nil) [ CSSD]2013-05-23 15:46:15.301 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 808 [ CSSD]2013-05-23 15:46:15.309 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:46:15.309 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c0e5900) skgpid ospid 17435 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:15.573 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:15.573 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014127, gin 1, member 0 [ CSSD]2013-05-23 15:46:15.573 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:46:15.573 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:46:15.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:15.579 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:46:15.579 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:46:15.789 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:16.093 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:16.093 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:16.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:16.120 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:17.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:18.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:18.126 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:19.805 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:20.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:20.138 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:20.232 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:46:20.232 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:46:20.233 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:46:20.233 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c0955a0) pid(17483) version 11:1:1:4 [ CSSD]2013-05-23 15:46:20.241 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:46:20.241 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:46:20.241 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 15:46:20.241 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:20.241 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 809 (0x7f431bfd8490) [ CSSD]2013-05-23 15:46:20.241 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8490 (RPC#809) tag(329002c) sent to node 1 [ CSSD]2013-05-23 15:46:20.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0329002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:20.241 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014128, gin 1, member 0 [ CSSD]2013-05-23 15:46:20.241 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:46:20.242 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:46:20.242 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:46:20.242 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8490 (RPC#809) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:20.242 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8490, ret 0, client 0x7f431c135120 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:46:20.242 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:46:20.242 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 809 [ CSSD]2013-05-23 15:46:20.248 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:46:20.248 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:20.248 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0959a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:20.248 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 810 (0x7f431bfd8510) [ CSSD]2013-05-23 15:46:20.248 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8510 (RPC#810) tag(32a002c) sent to node 1 [ CSSD]2013-05-23 15:46:20.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(032a002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:20.248 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:20.248 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8510 (RPC#810) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:20.248 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8510, ret 0, client 0x7f431c135120 member 0x7f431c0959a0 [ CSSD]2013-05-23 15:46:20.248 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 810 [ CSSD]2013-05-23 15:46:20.249 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0959a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:46:20.451 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:46:20.451 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c0e5900) skgpid ospid 17483 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:21.096 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:21.096 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:21.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:22.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:22.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:22.597 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 15:46:22.598 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:46:22.599 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 15:46:22.599 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c0955a0) pid(15782/15782) version 11:1:1:4 [ CSSD]2013-05-23 15:46:22.604 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c0955a0), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:46:22.604 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:46:22.604 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:46:22.604 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 15:46:22.969 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x231dc50) [ CSSD]2013-05-23 15:46:22.969 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x231dc50), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:46:22.969 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x231dc50), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:46:22.969 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 15:46:22.981 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ef070) [ CSSD]2013-05-23 15:46:22.981 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ef070), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:46:22.981 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ef070), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:46:22.981 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 15:46:22.981 [1191196992] >TRACE: clssgmDeadProc: proc 0x22da4d0 [ CSSD]2013-05-23 15:46:22.981 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22da4d0) con(0x22f3d50) skgpid 12035 ospid 12035 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:23.863 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:24.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:24.196 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:25.326 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:46:25.326 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 811 (0x7f431bfd8590) [ CSSD]2013-05-23 15:46:25.327 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8590 (RPC#811) tag(32b002c) sent to node 1 [ CSSD]2013-05-23 15:46:25.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:25.327 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 811 (0x7f431bfd8590) [ CSSD]2013-05-23 15:46:25.327 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8590 (RPC#811) state 6, flags 0x202 [ CSSD]2013-05-23 15:46:25.327 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8590, ret 0, client (nil) [ CSSD]2013-05-23 15:46:25.327 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 811 [ CSSD]2013-05-23 15:46:25.869 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:26.099 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:26.100 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:26.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:26.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:27.168 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c121750) allocated [ CSSD]2013-05-23 15:46:27.169 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:46:27.169 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c121750 - 1,2,3 [ CSSD]2013-05-23 15:46:27.169 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c121750) pid(17605/17605) version 11:1:1:4 [ CSSD]2013-05-23 15:46:27.175 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c121750), client(1/0x205ecb0) [ CSSD]2013-05-23 15:46:27.175 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x205ecb0) [ CSSD]2013-05-23 15:46:27.175 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:46:27.175 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 15:46:27.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:28.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:28.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:29.905 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:30.218 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:30.218 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:30.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:30.824 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014129, gin 1, member 0 [ CSSD]2013-05-23 15:46:30.824 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:46:30.824 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:46:30.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:30.831 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:46:30.831 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:46:31.102 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:31.103 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:31.910 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:32.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:32.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:33.963 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:34.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:34.296 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:35.358 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:46:35.358 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 812 (0x7f431bfd8610) [ CSSD]2013-05-23 15:46:35.359 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8610 (RPC#812) tag(32c002c) sent to node 1 [ CSSD]2013-05-23 15:46:35.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:35.359 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 812 (0x7f431bfd8610) [ CSSD]2013-05-23 15:46:35.359 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8610 (RPC#812) state 6, flags 0x202 [ CSSD]2013-05-23 15:46:35.359 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8610, ret 0, client (nil) [ CSSD]2013-05-23 15:46:35.359 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 812 [ CSSD]2013-05-23 15:46:35.482 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:46:35.483 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:46:35.484 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:46:35.484 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ed50) proc(0x22e01e0) pid(17700) version 11:1:1:4 [ CSSD]2013-05-23 15:46:35.500 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e01e0), client(1/0x2312950) [ CSSD]2013-05-23 15:46:35.500 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 15:46:35.500 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22ef060, requested num -1 [ CSSD]2013-05-23 15:46:35.500 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:35.500 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 813 (0x7f431bfd8690) [ CSSD]2013-05-23 15:46:35.501 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8690 (RPC#813) tag(32d002c) sent to node 1 [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(032d002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014130, gin 1, member 0 [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8690 (RPC#813) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8690, ret 0, client 0x2312950 member 0x22daa40 [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 15:46:35.501 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 813 [ CSSD]2013-05-23 15:46:35.509 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 15:46:35.509 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:35.509 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22daa40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:35.509 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 814 (0x7f431bfd8710) [ CSSD]2013-05-23 15:46:35.509 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8710 (RPC#814) tag(32e002c) sent to node 1 [ CSSD]2013-05-23 15:46:35.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(032e002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:35.509 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:35.510 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8710 (RPC#814) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:35.510 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8710, ret 0, client 0x2312950 member 0x22daa40 [ CSSD]2013-05-23 15:46:35.510 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 814 [ CSSD]2013-05-23 15:46:35.510 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22daa40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:46:35.712 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:46:35.712 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x205ed50) skgpid ospid 17700 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:35.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:36.106 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:36.106 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:36.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:36.322 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:37.981 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:38.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:38.327 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:40.006 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:40.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:40.339 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:40.950 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:46:40.951 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:46:40.952 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:46:40.952 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ed50) proc(0x22e01e0) pid(17603/17603) version 11:1:1:4 [ CSSD]2013-05-23 15:46:40.967 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e01e0), client(1/0x22ef070) [ CSSD]2013-05-23 15:46:40.967 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ef070) [ CSSD]2013-05-23 15:46:40.967 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:46:40.967 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 15:46:41.109 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:41.109 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:42.059 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:42.345 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:42.345 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:43.501 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c121750), client(2/0x22e2eb0) [ CSSD]2013-05-23 15:46:43.501 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22e2eb0) [ CSSD]2013-05-23 15:46:43.501 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:46:43.501 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 15:46:44.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:44.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:44.347 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:45.384 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:46:45.384 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 815 (0x7f431bfd8790) [ CSSD]2013-05-23 15:46:45.385 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8790 (RPC#815) tag(32f002c) sent to node 1 [ CSSD]2013-05-23 15:46:45.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:45.385 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 815 (0x7f431bfd8790) [ CSSD]2013-05-23 15:46:45.385 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8790 (RPC#815) state 6, flags 0x202 [ CSSD]2013-05-23 15:46:45.385 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8790, ret 0, client (nil) [ CSSD]2013-05-23 15:46:45.385 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 815 [ CSSD]2013-05-23 15:46:46.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:46.073 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014131, gin 1, member 0 [ CSSD]2013-05-23 15:46:46.074 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:46:46.074 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:46:46.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:46.081 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:46:46.081 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7420) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:46:46.089 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:46.112 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:46.113 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:46.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:46.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:48.102 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:48.379 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:48.379 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:50.107 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:50.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:50.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:50.744 [1191196992] >TRACE: clssgmAllocProc: (0x22f3c80) allocated [ CSSD]2013-05-23 15:46:50.745 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:46:50.745 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3c80 - 1,2,3 [ CSSD]2013-05-23 15:46:50.745 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x22f3c80) pid(17846) version 11:1:1:4 [ CSSD]2013-05-23 15:46:50.750 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f3c80), client(1/0x22fe460) [ CSSD]2013-05-23 15:46:50.751 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fe460) [ CSSD]2013-05-23 15:46:50.751 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fe460 with con 0x22dfed0, requested num -1 [ CSSD]2013-05-23 15:46:50.751 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:50.751 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 816 (0x7f431bfd8810) [ CSSD]2013-05-23 15:46:50.751 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8810 (RPC#816) tag(330002c) sent to node 1 [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0330002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014132, gin 1, member 0 [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8810 (RPC#816) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8810, ret 0, client 0x22fe460 member 0x22fe5d0 [ CSSD]2013-05-23 15:46:50.751 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fe460 [ CSSD]2013-05-23 15:46:50.752 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 816 [ CSSD]2013-05-23 15:46:50.757 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fe460) [ CSSD]2013-05-23 15:46:50.757 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fe460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:50.757 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fe5d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:46:50.757 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 817 (0x7f431bfd8890) [ CSSD]2013-05-23 15:46:50.757 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8890 (RPC#817) tag(331002c) sent to node 1 [ CSSD]2013-05-23 15:46:50.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0331002c) incarnation(174631051) [ CSSD]2013-05-23 15:46:50.758 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:46:50.758 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8890 (RPC#817) state 6, flags 0x100 [ CSSD]2013-05-23 15:46:50.758 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8890, ret 0, client 0x22fe460 member 0x22fe5d0 [ CSSD]2013-05-23 15:46:50.758 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 817 [ CSSD]2013-05-23 15:46:50.758 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fe5d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:46:50.960 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3c80 [ CSSD]2013-05-23 15:46:50.961 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3c80) con(0x22dacd0) skgpid ospid 17846 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:46:51.115 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:51.116 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:52.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:52.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:52.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:54.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:54.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:54.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:55.423 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:46:55.424 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 818 (0x7f431bfd8910) [ CSSD]2013-05-23 15:46:55.424 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8910 (RPC#818) tag(332002c) sent to node 1 [ CSSD]2013-05-23 15:46:55.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:55.424 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 818 (0x7f431bfd8910) [ CSSD]2013-05-23 15:46:55.424 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8910 (RPC#818) state 6, flags 0x202 [ CSSD]2013-05-23 15:46:55.424 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8910, ret 0, client (nil) [ CSSD]2013-05-23 15:46:55.424 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 818 [ CSSD]2013-05-23 15:46:56.119 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:46:56.119 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:46:56.189 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:56.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:56.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:46:58.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:46:58.505 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:46:58.505 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:00.208 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:00.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:00.529 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:01.122 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:01.123 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:01.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:01.334 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014133, gin 1, member 0 [ CSSD]2013-05-23 15:47:01.334 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:01.334 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:01.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:01.341 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:01.341 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7420) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:02.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:02.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:02.535 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:04.266 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:04.547 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:04.547 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:05.447 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:47:05.447 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 819 (0x7f431bfd8990) [ CSSD]2013-05-23 15:47:05.448 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8990 (RPC#819) tag(333002c) sent to node 1 [ CSSD]2013-05-23 15:47:05.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:05.448 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 819 (0x7f431bfd8990) [ CSSD]2013-05-23 15:47:05.448 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8990 (RPC#819) state 6, flags 0x202 [ CSSD]2013-05-23 15:47:05.448 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8990, ret 0, client (nil) [ CSSD]2013-05-23 15:47:05.448 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 819 [ CSSD]2013-05-23 15:47:05.993 [1191196992] >TRACE: clssgmAllocProc: (0x22f3c80) allocated [ CSSD]2013-05-23 15:47:05.994 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:47:05.994 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3c80 - 1,2,3 [ CSSD]2013-05-23 15:47:05.994 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x22f3c80) pid(17994) version 11:1:1:4 [ CSSD]2013-05-23 15:47:06.010 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f3c80), client(1/0x22fe460) [ CSSD]2013-05-23 15:47:06.010 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fe460) [ CSSD]2013-05-23 15:47:06.010 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fe460 with con 0x22dfed0, requested num -1 [ CSSD]2013-05-23 15:47:06.010 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:06.010 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 820 (0x7f431bfd8a10) [ CSSD]2013-05-23 15:47:06.011 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8a10 (RPC#820) tag(334002c) sent to node 1 [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0334002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014134, gin 1, member 0 [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8a10 (RPC#820) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8a10, ret 0, client 0x22fe460 member 0x22fe5d0 [ CSSD]2013-05-23 15:47:06.011 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fe460 [ CSSD]2013-05-23 15:47:06.012 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 820 [ CSSD]2013-05-23 15:47:06.018 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fe460) [ CSSD]2013-05-23 15:47:06.018 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fe460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:06.018 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fe5d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:06.018 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 821 (0x7f431bfd8a90) [ CSSD]2013-05-23 15:47:06.018 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8a90 (RPC#821) tag(335002c) sent to node 1 [ CSSD]2013-05-23 15:47:06.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0335002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:06.018 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:06.018 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8a90 (RPC#821) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:06.018 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8a90, ret 0, client 0x22fe460 member 0x22fe5d0 [ CSSD]2013-05-23 15:47:06.019 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 821 [ CSSD]2013-05-23 15:47:06.019 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fe5d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:47:06.126 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:06.126 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:06.221 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3c80 [ CSSD]2013-05-23 15:47:06.221 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3c80) con(0x22dacd0) skgpid ospid 17994 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:06.270 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:06.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:06.554 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:06.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:06.704 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014135, gin 1, member 0 [ CSSD]2013-05-23 15:47:06.704 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:06.704 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:06.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:06.710 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:06.710 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7420) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:08.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:08.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:08.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:10.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:10.453 [1191196992] >TRACE: clssgmAllocProc: (0x22f3c80) allocated [ CSSD]2013-05-23 15:47:10.454 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:47:10.454 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3c80 - 1,2,3 [ CSSD]2013-05-23 15:47:10.455 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dad90) proc(0x22f3c80) pid(18061) version 11:1:1:4 [ CSSD]2013-05-23 15:47:10.461 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f3c80), client(1/0x22fe660) [ CSSD]2013-05-23 15:47:10.461 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fe660) [ CSSD]2013-05-23 15:47:10.461 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fe660 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 15:47:10.461 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:10.461 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 822 (0x7f431bfd8b10) [ CSSD]2013-05-23 15:47:10.461 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8b10 (RPC#822) tag(336002c) sent to node 1 [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0336002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014136, gin 1, member 0 [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8b10 (RPC#822) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8b10, ret 0, client 0x22fe660 member 0x22dfe20 [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fe660 [ CSSD]2013-05-23 15:47:10.462 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 822 [ CSSD]2013-05-23 15:47:10.468 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fe660) [ CSSD]2013-05-23 15:47:10.468 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fe660), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:10.468 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22dfe20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:10.468 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 823 (0x7f431bfd8b90) [ CSSD]2013-05-23 15:47:10.468 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8b90 (RPC#823) tag(337002c) sent to node 1 [ CSSD]2013-05-23 15:47:10.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0337002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:10.469 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:10.469 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8b90 (RPC#823) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:10.469 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8b90, ret 0, client 0x22fe660 member 0x22dfe20 [ CSSD]2013-05-23 15:47:10.469 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 823 [ CSSD]2013-05-23 15:47:10.469 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22dfe20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:47:10.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:10.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:10.671 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3c80 [ CSSD]2013-05-23 15:47:10.671 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3c80) con(0x22dad90) skgpid ospid 18061 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:10.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:10.863 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014137, gin 1, member 0 [ CSSD]2013-05-23 15:47:10.864 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:10.864 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:10.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:10.871 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:10.871 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7420) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:11.129 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:11.129 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:12.361 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:12.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:12.635 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:13.393 [1191196992] >TRACE: clssgmAllocProc: (0x22f3c80) allocated [ CSSD]2013-05-23 15:47:13.394 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:47:13.394 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3c80 - 1,2,3 [ CSSD]2013-05-23 15:47:13.394 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x22f3c80) pid(18214) version 11:1:1:4 [ CSSD]2013-05-23 15:47:13.542 [1191196992] >TRACE: clssgmAllocProc: (0x22fe660) allocated [ CSSD]2013-05-23 15:47:13.543 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:47:13.544 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fe660 - 1,2,3 [ CSSD]2013-05-23 15:47:13.544 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x22fe660) pid(18237) version 11:1:1:4 [ CSSD]2013-05-23 15:47:13.560 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:47:13.561 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:47:13.564 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fe660 [ CSSD]2013-05-23 15:47:13.564 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fe660) con(0x22dffa0) skgpid ospid 18237 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:13.639 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(60/0x22fe460) [ CSSD]2013-05-23 15:47:13.639 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 60 (0x22fe460) [ CSSD]2013-05-23 15:47:13.639 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22fe460 with con 0x22dfed0, requested num -1 [ CSSD]2013-05-23 15:47:13.639 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:47:13.639 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 824 (0x7f431bfd8c10) [ CSSD]2013-05-23 15:47:13.639 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8c10 (RPC#824) tag(338002c) sent to node 1 [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0338002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014138, gin 1, member 0 [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x205e170/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8c10 (RPC#824) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8c10, ret 0, client 0x22fe460 member 0x22fe5d0 [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fe460 [ CSSD]2013-05-23 15:47:13.640 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 824 [ CSSD]2013-05-23 15:47:13.646 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 60 (0x22fe460) [ CSSD]2013-05-23 15:47:13.646 [1191196992] >TRACE: clssgmExitGrock: client 60 (0x22fe460), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:47:13.646 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fe5d0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:47:13.646 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 825 (0x7f431bfd8c90) [ CSSD]2013-05-23 15:47:13.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8c90 (RPC#825) tag(339002c) sent to node 1 [ CSSD]2013-05-23 15:47:13.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0339002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:13.647 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:47:13.647 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8c90 (RPC#825) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:13.647 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8c90, ret 0, client 0x22fe460 member 0x22fe5d0 [ CSSD]2013-05-23 15:47:13.647 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 825 [ CSSD]2013-05-23 15:47:13.647 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22fe5d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:47:13.648 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3c80 [ CSSD]2013-05-23 15:47:13.648 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3c80) con(0x22dacd0) skgpid ospid 18214 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:14.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:14.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:14.728 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:15.469 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:47:15.470 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 826 (0x7f431bfd8d10) [ CSSD]2013-05-23 15:47:15.470 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8d10 (RPC#826) tag(33a002c) sent to node 1 [ CSSD]2013-05-23 15:47:15.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:15.470 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 826 (0x7f431bfd8d10) [ CSSD]2013-05-23 15:47:15.470 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8d10 (RPC#826) state 6, flags 0x202 [ CSSD]2013-05-23 15:47:15.470 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8d10, ret 0, client (nil) [ CSSD]2013-05-23 15:47:15.470 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 826 [ CSSD]2013-05-23 15:47:16.132 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:16.132 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:16.371 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:16.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:16.587 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014139, gin 1, member 0 [ CSSD]2013-05-23 15:47:16.587 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:16.587 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:16.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:16.594 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:16.594 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:16.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:16.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:18.463 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:18.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:18.746 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:20.469 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:20.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:20.772 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:21.135 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:21.135 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:21.254 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090cf0) allocated [ CSSD]2013-05-23 15:47:21.255 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:47:21.255 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090cf0 - 1,2,3 [ CSSD]2013-05-23 15:47:21.255 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c090cf0) pid(18389) version 11:1:1:4 [ CSSD]2013-05-23 15:47:21.271 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c090cf0), client(1/0x7f431c13f730) [ CSSD]2013-05-23 15:47:21.271 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:47:21.271 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c090ef0, requested num -1 [ CSSD]2013-05-23 15:47:21.271 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:21.271 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 827 (0x7f431bfd8d90) [ CSSD]2013-05-23 15:47:21.271 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8d90 (RPC#827) tag(33b002c) sent to node 1 [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(033b002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014140, gin 1, member 0 [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0914c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8d90 (RPC#827) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8d90, ret 0, client 0x7f431c13f730 member 0x7f431c124520 [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 15:47:21.272 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 827 [ CSSD]2013-05-23 15:47:21.279 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:47:21.279 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:21.279 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c124520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:21.279 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 828 (0x7f431bfd8e10) [ CSSD]2013-05-23 15:47:21.279 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8e10 (RPC#828) tag(33c002c) sent to node 1 [ CSSD]2013-05-23 15:47:21.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(033c002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:21.279 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:21.279 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8e10 (RPC#828) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:21.279 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8e10, ret 0, client 0x7f431c13f730 member 0x7f431c124520 [ CSSD]2013-05-23 15:47:21.280 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 828 [ CSSD]2013-05-23 15:47:21.280 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c124520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:47:21.482 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090cf0 [ CSSD]2013-05-23 15:47:21.482 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090cf0) con(0x7f431c12f0c0) skgpid ospid 18389 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:22.481 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:22.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:22.797 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:24.505 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:24.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:24.809 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:25.513 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:47:25.514 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 829 (0x7f431bfd8e90) [ CSSD]2013-05-23 15:47:25.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8e90 (RPC#829) tag(33d002c) sent to node 1 [ CSSD]2013-05-23 15:47:25.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:25.514 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 829 (0x7f431bfd8e90) [ CSSD]2013-05-23 15:47:25.514 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8e90 (RPC#829) state 6, flags 0x202 [ CSSD]2013-05-23 15:47:25.514 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8e90, ret 0, client (nil) [ CSSD]2013-05-23 15:47:25.514 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 829 [ CSSD]2013-05-23 15:47:26.138 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:26.138 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:26.511 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:26.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:26.815 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:28.514 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:28.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:28.826 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:30.520 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:30.832 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:30.832 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:31.141 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:31.142 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:31.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:31.848 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014141, gin 1, member 0 [ CSSD]2013-05-23 15:47:31.848 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:31.848 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:31.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:31.856 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:31.856 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:32.571 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:32.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:32.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:34.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:34.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:34.928 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:35.144 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:35.144 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:47:35.539 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:47:35.540 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 830 (0x7f431bfd8f10) [ CSSD]2013-05-23 15:47:35.540 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8f10 (RPC#830) tag(33e002c) sent to node 1 [ CSSD]2013-05-23 15:47:35.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:35.540 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 830 (0x7f431bfd8f10) [ CSSD]2013-05-23 15:47:35.540 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8f10 (RPC#830) state 6, flags 0x202 [ CSSD]2013-05-23 15:47:35.540 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8f10, ret 0, client (nil) [ CSSD]2013-05-23 15:47:35.540 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 830 [ CSSD]2013-05-23 15:47:36.515 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090cf0) allocated [ CSSD]2013-05-23 15:47:36.516 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:47:36.516 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090cf0 - 1,2,3 [ CSSD]2013-05-23 15:47:36.517 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c090cf0) pid(18546) version 11:1:1:4 [ CSSD]2013-05-23 15:47:36.532 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c090cf0), client(1/0x7f431c135120) [ CSSD]2013-05-23 15:47:36.532 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:47:36.532 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c090ef0, requested num -1 [ CSSD]2013-05-23 15:47:36.532 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:36.532 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 831 (0x7f431bfd8f90) [ CSSD]2013-05-23 15:47:36.532 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8f90 (RPC#831) tag(33f002c) sent to node 1 [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(033f002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014142, gin 1, member 0 [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0914c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8f90 (RPC#831) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8f90, ret 0, client 0x7f431c135120 member 0x7f431c124520 [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 15:47:36.533 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 831 [ CSSD]2013-05-23 15:47:36.539 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 15:47:36.539 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:36.539 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c124520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:36.539 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 832 (0x7f431bfd9010) [ CSSD]2013-05-23 15:47:36.539 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9010 (RPC#832) tag(340002c) sent to node 1 [ CSSD]2013-05-23 15:47:36.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0340002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:36.539 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:36.539 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9010 (RPC#832) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:36.539 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9010, ret 0, client 0x7f431c135120 member 0x7f431c124520 [ CSSD]2013-05-23 15:47:36.540 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 832 [ CSSD]2013-05-23 15:47:36.540 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c124520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:47:36.583 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:36.742 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090cf0 [ CSSD]2013-05-23 15:47:36.742 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090cf0) con(0x7f431c12f0c0) skgpid ospid 18546 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:36.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:36.934 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:38.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:38.945 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:38.945 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:40.148 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:40.148 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:40.378 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:40.378 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014143, gin 1, member 0 [ CSSD]2013-05-23 15:47:40.378 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:40.378 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:40.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:40.384 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:40.384 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:40.599 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:40.951 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:40.951 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:42.672 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:42.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:42.957 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:44.697 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:45.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:45.009 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:45.151 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:45.151 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:45.565 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:47:45.565 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 833 (0x7f431bfd9090) [ CSSD]2013-05-23 15:47:45.565 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9090 (RPC#833) tag(341002c) sent to node 1 [ CSSD]2013-05-23 15:47:45.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:45.566 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 833 (0x7f431bfd9090) [ CSSD]2013-05-23 15:47:45.566 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9090 (RPC#833) state 6, flags 0x202 [ CSSD]2013-05-23 15:47:45.566 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9090, ret 0, client (nil) [ CSSD]2013-05-23 15:47:45.566 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 833 [ CSSD]2013-05-23 15:47:46.703 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:47.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:47.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:47.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:47.098 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014144, gin 1, member 0 [ CSSD]2013-05-23 15:47:47.098 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:47:47.098 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:47:47.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:47.105 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:47:47.105 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:47:48.660 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22dabf0) [ CSSD]2013-05-23 15:47:48.660 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22dabf0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:47:48.660 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22dabf0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:47:48.660 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 15:47:48.670 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f3be0) [ CSSD]2013-05-23 15:47:48.670 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f3be0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:47:48.670 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f3be0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:47:48.670 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 15:47:48.670 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ff460 [ CSSD]2013-05-23 15:47:48.670 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ff460) con(0x22eeea0) skgpid 13244 ospid 13244 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:48.713 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:49.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:49.017 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:50.154 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:50.155 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:47:50.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:51.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:51.023 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:51.773 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090cf0) allocated [ CSSD]2013-05-23 15:47:51.775 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:47:51.775 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090cf0 - 1,2,3 [ CSSD]2013-05-23 15:47:51.775 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c090cf0) pid(18763) version 11:1:1:4 [ CSSD]2013-05-23 15:47:51.792 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c090cf0), client(1/0x7f431c108e70) [ CSSD]2013-05-23 15:47:51.792 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:47:51.792 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c090ef0, requested num -1 [ CSSD]2013-05-23 15:47:51.792 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:51.792 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 834 (0x7f431bfd9110) [ CSSD]2013-05-23 15:47:51.792 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9110 (RPC#834) tag(342002c) sent to node 1 [ CSSD]2013-05-23 15:47:51.792 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0342002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:51.792 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014145, gin 1, member 0 [ CSSD]2013-05-23 15:47:51.792 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0914b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:47:51.793 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:47:51.793 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:47:51.793 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9110 (RPC#834) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:51.793 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9110, ret 0, client 0x7f431c108e70 member 0x7f431c124520 [ CSSD]2013-05-23 15:47:51.793 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 15:47:51.793 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 834 [ CSSD]2013-05-23 15:47:51.799 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 15:47:51.799 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:51.799 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c124520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:47:51.799 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 835 (0x7f431bfd9190) [ CSSD]2013-05-23 15:47:51.799 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9190 (RPC#835) tag(343002c) sent to node 1 [ CSSD]2013-05-23 15:47:51.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0343002c) incarnation(174631051) [ CSSD]2013-05-23 15:47:51.800 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:47:51.800 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9190 (RPC#835) state 6, flags 0x100 [ CSSD]2013-05-23 15:47:51.800 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9190, ret 0, client 0x7f431c108e70 member 0x7f431c124520 [ CSSD]2013-05-23 15:47:51.800 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 835 [ CSSD]2013-05-23 15:47:51.800 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c124520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:47:52.003 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090cf0 [ CSSD]2013-05-23 15:47:52.003 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090cf0) con(0x7f431c12f0c0) skgpid ospid 18763 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:47:52.772 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:53.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:53.048 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:53.985 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090cf0) allocated [ CSSD]2013-05-23 15:47:53.986 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:47:53.986 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090cf0 - 1,2,3 [ CSSD]2013-05-23 15:47:53.986 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c090cf0) pid(18743/18743) version 11:1:1:4 [ CSSD]2013-05-23 15:47:54.002 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c090cf0), client(1/0x22ff530) [ CSSD]2013-05-23 15:47:54.002 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ff530) [ CSSD]2013-05-23 15:47:54.002 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:47:54.002 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 15:47:54.157 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:54.157 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:47:54.778 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:55.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:55.059 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:55.589 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:47:55.589 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 836 (0x7f431bfd9210) [ CSSD]2013-05-23 15:47:55.589 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9210 (RPC#836) tag(344002c) sent to node 1 [ CSSD]2013-05-23 15:47:55.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:55.590 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 836 (0x7f431bfd9210) [ CSSD]2013-05-23 15:47:55.590 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9210 (RPC#836) state 6, flags 0x202 [ CSSD]2013-05-23 15:47:55.590 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9210, ret 0, client (nil) [ CSSD]2013-05-23 15:47:55.590 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 836 [ CSSD]2013-05-23 15:47:55.662 [1191196992] >TRACE: clssgmAllocProc: (0x2302b40) allocated [ CSSD]2013-05-23 15:47:55.662 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:47:55.663 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302b40 - 1,2,3 [ CSSD]2013-05-23 15:47:55.663 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x2302b40) pid(18741/18741) version 11:1:1:4 [ CSSD]2013-05-23 15:47:55.666 [1191196992] >TRACE: clssgmAllocProc: (0x22d7620) allocated [ CSSD]2013-05-23 15:47:55.666 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:47:55.667 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7620 - 1,2,3 [ CSSD]2013-05-23 15:47:55.667 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x22d7620) pid(18739/18739) version 11:1:1:4 [ CSSD]2013-05-23 15:47:55.688 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2302b40), client(1/0x22ff8b0) [ CSSD]2013-05-23 15:47:55.688 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ff8b0) [ CSSD]2013-05-23 15:47:55.688 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:47:55.688 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 15:47:55.689 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d7620), client(1/0x2302dc0) [ CSSD]2013-05-23 15:47:55.689 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2302dc0) [ CSSD]2013-05-23 15:47:55.689 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:47:55.689 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 15:47:56.785 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:57.065 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:57.065 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:57.989 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d7620), client(2/0x22f4160) [ CSSD]2013-05-23 15:47:57.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f4160) [ CSSD]2013-05-23 15:47:57.989 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:47:57.989 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 15:47:58.191 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e01e0), client(2/0x22f4200) [ CSSD]2013-05-23 15:47:58.191 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f4200) [ CSSD]2013-05-23 15:47:58.191 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:47:58.191 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 15:47:58.307 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2302b40), client(2/0x23024d0) [ CSSD]2013-05-23 15:47:58.307 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x23024d0) [ CSSD]2013-05-23 15:47:58.308 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:47:58.308 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 15:47:58.461 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c090cf0), client(2/0x22f7f80) [ CSSD]2013-05-23 15:47:58.461 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f7f80) [ CSSD]2013-05-23 15:47:58.462 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:47:58.462 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 28 [ CSSD]2013-05-23 15:47:58.795 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:47:59.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:47:59.067 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:47:59.160 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:47:59.161 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:00.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:01.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:01.073 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:02.350 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:02.350 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014146, gin 1, member 0 [ CSSD]2013-05-23 15:48:02.350 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:48:02.350 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:48:02.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:02.357 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:48:02.357 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f83f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:48:02.893 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:03.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:03.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:04.164 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:04.164 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:04.898 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:05.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:05.130 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:05.613 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:48:05.613 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 837 (0x7f431bfd9290) [ CSSD]2013-05-23 15:48:05.613 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9290 (RPC#837) tag(345002c) sent to node 1 [ CSSD]2013-05-23 15:48:05.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:05.614 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 837 (0x7f431bfd9290) [ CSSD]2013-05-23 15:48:05.614 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9290 (RPC#837) state 6, flags 0x202 [ CSSD]2013-05-23 15:48:05.614 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9290, ret 0, client (nil) [ CSSD]2013-05-23 15:48:05.614 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 837 [ CSSD]2013-05-23 15:48:06.904 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:06.961 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:06.961 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014147, gin 1, member 0 [ CSSD]2013-05-23 15:48:06.961 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f8200/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:48:06.961 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:48:06.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:06.968 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:48:06.968 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f7e50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:48:07.044 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 15:48:07.045 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:48:07.045 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 15:48:07.045 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23233c0) proc(0x231ed60) pid(18928) version 11:1:1:4 [ CSSD]2013-05-23 15:48:07.051 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x231ed60), client(1/0x22f1b90) [ CSSD]2013-05-23 15:48:07.051 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1b90) [ CSSD]2013-05-23 15:48:07.051 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1b90 with con 0x2324290, requested num -1 [ CSSD]2013-05-23 15:48:07.051 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:07.051 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 838 (0x7f431bfd9310) [ CSSD]2013-05-23 15:48:07.051 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9310 (RPC#838) tag(346002c) sent to node 1 [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0346002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014148, gin 1, member 0 [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f1d00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9310 (RPC#838) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9310, ret 0, client 0x22f1b90 member 0x2324060 [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1b90 [ CSSD]2013-05-23 15:48:07.052 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 838 [ CSSD]2013-05-23 15:48:07.058 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1b90) [ CSSD]2013-05-23 15:48:07.058 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1b90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:07.058 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2324060) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:07.058 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 839 (0x7f431bfd9390) [ CSSD]2013-05-23 15:48:07.058 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9390 (RPC#839) tag(347002c) sent to node 1 [ CSSD]2013-05-23 15:48:07.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0347002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:07.059 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:07.059 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9390 (RPC#839) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:07.059 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9390, ret 0, client 0x22f1b90 member 0x2324060 [ CSSD]2013-05-23 15:48:07.059 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 839 [ CSSD]2013-05-23 15:48:07.059 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2324060) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:48:07.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:07.136 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:07.261 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 15:48:07.261 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x23233c0) skgpid ospid 18928 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:48:08.916 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:09.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:09.148 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:09.167 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:09.167 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:10.713 [1191196992] >TRACE: clssgmAllocProc: (0x22f7e40) allocated [ CSSD]2013-05-23 15:48:10.714 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:48:10.714 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f7e40 - 1,2,3 [ CSSD]2013-05-23 15:48:10.714 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2323490) proc(0x22f7e40) pid(18995) version 11:1:1:4 [ CSSD]2013-05-23 15:48:10.720 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f7e40), client(1/0x231ee10) [ CSSD]2013-05-23 15:48:10.720 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231ee10) [ CSSD]2013-05-23 15:48:10.720 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231ee10 with con 0x231ed60, requested num -1 [ CSSD]2013-05-23 15:48:10.720 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:10.720 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 840 (0x7f431bfd9410) [ CSSD]2013-05-23 15:48:10.720 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9410 (RPC#840) tag(348002c) sent to node 1 [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0348002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014149, gin 1, member 0 [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f1c60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9410 (RPC#840) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9410, ret 0, client 0x231ee10 member 0x2324130 [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231ee10 [ CSSD]2013-05-23 15:48:10.721 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 840 [ CSSD]2013-05-23 15:48:10.728 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231ee10) [ CSSD]2013-05-23 15:48:10.728 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231ee10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:10.728 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2324130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:10.728 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 841 (0x7f431bfd9490) [ CSSD]2013-05-23 15:48:10.728 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9490 (RPC#841) tag(349002c) sent to node 1 [ CSSD]2013-05-23 15:48:10.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0349002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:10.729 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:10.729 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9490 (RPC#841) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:10.729 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9490, ret 0, client 0x231ee10 member 0x2324130 [ CSSD]2013-05-23 15:48:10.729 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 841 [ CSSD]2013-05-23 15:48:10.729 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2324130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:48:10.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:10.931 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f7e40 [ CSSD]2013-05-23 15:48:10.931 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f7e40) con(0x2323490) skgpid ospid 18995 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:48:11.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:11.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:12.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:13.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:13.158 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:13.169 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:13.169 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:48:14.977 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:15.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:15.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:15.639 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:48:15.640 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 842 (0x7f431bfd9510) [ CSSD]2013-05-23 15:48:15.640 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9510 (RPC#842) tag(34a002c) sent to node 1 [ CSSD]2013-05-23 15:48:15.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:15.640 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 842 (0x7f431bfd9510) [ CSSD]2013-05-23 15:48:15.640 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9510 (RPC#842) state 6, flags 0x202 [ CSSD]2013-05-23 15:48:15.640 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9510, ret 0, client (nil) [ CSSD]2013-05-23 15:48:15.641 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 842 [ CSSD]2013-05-23 15:48:16.983 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:17.176 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:17.176 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:17.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:17.600 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014150, gin 1, member 0 [ CSSD]2013-05-23 15:48:17.600 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:48:17.600 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:48:17.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:17.606 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:48:17.606 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f83f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:48:18.172 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:18.173 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:19.075 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:19.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:19.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:21.080 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:21.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:21.253 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:22.294 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 15:48:22.294 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:48:22.295 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 15:48:22.295 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23235b0) proc(0x231ed60) pid(19101) version 11:1:1:4 [ CSSD]2013-05-23 15:48:22.301 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x231ed60), client(1/0x23242e0) [ CSSD]2013-05-23 15:48:22.301 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23242e0) [ CSSD]2013-05-23 15:48:22.301 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23242e0 with con 0x22f1730, requested num -1 [ CSSD]2013-05-23 15:48:22.301 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:22.301 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 843 (0x7f431bfd9590) [ CSSD]2013-05-23 15:48:22.301 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9590 (RPC#843) tag(34b002c) sent to node 1 [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(034b002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014151, gin 1, member 0 [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f1d10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9590 (RPC#843) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9590, ret 0, client 0x23242e0 member 0x22f2010 [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23242e0 [ CSSD]2013-05-23 15:48:22.302 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 843 [ CSSD]2013-05-23 15:48:22.308 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23242e0) [ CSSD]2013-05-23 15:48:22.308 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23242e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:22.308 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2010) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:22.308 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 844 (0x7f431bfd9610) [ CSSD]2013-05-23 15:48:22.308 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9610 (RPC#844) tag(34c002c) sent to node 1 [ CSSD]2013-05-23 15:48:22.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(034c002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:22.308 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:22.308 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9610 (RPC#844) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:22.308 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9610, ret 0, client 0x23242e0 member 0x22f2010 [ CSSD]2013-05-23 15:48:22.309 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 844 [ CSSD]2013-05-23 15:48:22.309 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2010) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:48:22.511 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 15:48:22.511 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x23235b0) skgpid ospid 19101 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:48:23.093 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:23.175 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:23.175 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:23.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:23.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:25.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:25.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:25.271 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:25.692 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:48:25.692 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 845 (0x7f431bfd9690) [ CSSD]2013-05-23 15:48:25.693 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9690 (RPC#845) tag(34d002c) sent to node 1 [ CSSD]2013-05-23 15:48:25.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:25.693 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 845 (0x7f431bfd9690) [ CSSD]2013-05-23 15:48:25.693 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9690 (RPC#845) state 6, flags 0x202 [ CSSD]2013-05-23 15:48:25.693 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9690, ret 0, client (nil) [ CSSD]2013-05-23 15:48:25.693 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 845 [ CSSD]2013-05-23 15:48:27.191 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:27.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:27.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:28.178 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:28.178 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:29.196 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:29.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:29.329 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:31.201 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:31.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:31.335 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:32.180 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:32.181 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:48:32.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:32.848 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014152, gin 1, member 0 [ CSSD]2013-05-23 15:48:32.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:48:32.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:48:32.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:32.855 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:48:32.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f83f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:48:33.214 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:33.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:33.340 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:35.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:35.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:35.351 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:35.722 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:48:35.722 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 846 (0x7f431bfd9710) [ CSSD]2013-05-23 15:48:35.722 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9710 (RPC#846) tag(34e002c) sent to node 1 [ CSSD]2013-05-23 15:48:35.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:35.723 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 846 (0x7f431bfd9710) [ CSSD]2013-05-23 15:48:35.723 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9710 (RPC#846) state 6, flags 0x202 [ CSSD]2013-05-23 15:48:35.723 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9710, ret 0, client (nil) [ CSSD]2013-05-23 15:48:35.723 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 846 [ CSSD]2013-05-23 15:48:36.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:36.268 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014153, gin 1, member 0 [ CSSD]2013-05-23 15:48:36.268 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 15:48:36.268 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:48:36.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:36.274 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 15:48:36.274 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x2323300) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:48:36.404 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:48:36.405 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:48:36.406 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:48:36.407 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:48:36.408 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:48:36.408 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:48:37.183 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:37.184 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:37.222 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:37.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:37.357 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:37.543 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136f00) allocated [ CSSD]2013-05-23 15:48:37.544 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:48:37.544 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136f00 - 1,2,3 [ CSSD]2013-05-23 15:48:37.544 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090ef0) proc(0x7f431c136f00) pid(19244) version 11:1:1:4 [ CSSD]2013-05-23 15:48:37.550 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c136f00), client(1/0x7f431c13f730) [ CSSD]2013-05-23 15:48:37.550 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:48:37.550 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c137160, requested num -1 [ CSSD]2013-05-23 15:48:37.550 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:37.550 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 847 (0x7f431bfd9790) [ CSSD]2013-05-23 15:48:37.550 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9790 (RPC#847) tag(34f002c) sent to node 1 [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(034f002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014154, gin 1, member 0 [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137740/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9790 (RPC#847) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9790, ret 0, client 0x7f431c13f730 member 0x7f431c136de0 [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 15:48:37.551 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 847 [ CSSD]2013-05-23 15:48:37.558 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:48:37.558 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:37.558 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:37.558 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 848 (0x7f431bfd9810) [ CSSD]2013-05-23 15:48:37.558 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9810 (RPC#848) tag(350002c) sent to node 1 [ CSSD]2013-05-23 15:48:37.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0350002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:37.558 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:37.558 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9810 (RPC#848) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:37.558 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9810, ret 0, client 0x7f431c13f730 member 0x7f431c136de0 [ CSSD]2013-05-23 15:48:37.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 848 [ CSSD]2013-05-23 15:48:37.558 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:48:37.761 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136f00 [ CSSD]2013-05-23 15:48:37.761 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136f00) con(0x7f431c090ef0) skgpid ospid 19244 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:48:39.227 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:39.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:39.388 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:41.233 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:41.414 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:41.414 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:42.187 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:42.187 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:43.284 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:43.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:43.419 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:45.290 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:45.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:45.432 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:45.747 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:48:45.747 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 849 (0x7f431bfd9890) [ CSSD]2013-05-23 15:48:45.747 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9890 (RPC#849) tag(351002c) sent to node 1 [ CSSD]2013-05-23 15:48:45.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:45.748 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 849 (0x7f431bfd9890) [ CSSD]2013-05-23 15:48:45.748 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9890 (RPC#849) state 6, flags 0x202 [ CSSD]2013-05-23 15:48:45.748 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9890, ret 0, client (nil) [ CSSD]2013-05-23 15:48:45.748 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 849 [ CSSD]2013-05-23 15:48:47.190 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:47.190 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:47.382 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:47.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:47.477 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:48.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:48.096 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014155, gin 1, member 0 [ CSSD]2013-05-23 15:48:48.096 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:48:48.096 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:48:48.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:48.102 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:48:48.102 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f7d80) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:48:49.388 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:49.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:49.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:51.394 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:51.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:51.555 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:52.193 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:52.194 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:48:52.793 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136f00) allocated [ CSSD]2013-05-23 15:48:52.794 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:48:52.794 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136f00 - 1,2,3 [ CSSD]2013-05-23 15:48:52.794 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091480) proc(0x7f431c136f00) pid(19400) version 11:1:1:4 [ CSSD]2013-05-23 15:48:52.800 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c136f00), client(1/0x7f431c095b30) [ CSSD]2013-05-23 15:48:52.800 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 15:48:52.800 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095b30 with con 0x7f431c137160, requested num -1 [ CSSD]2013-05-23 15:48:52.800 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:52.800 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 850 (0x7f431bfd9910) [ CSSD]2013-05-23 15:48:52.801 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9910 (RPC#850) tag(352002c) sent to node 1 [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0352002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014156, gin 1, member 0 [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137740/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9910 (RPC#850) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9910, ret 0, client 0x7f431c095b30 member 0x7f431c136de0 [ CSSD]2013-05-23 15:48:52.801 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095b30 [ CSSD]2013-05-23 15:48:52.802 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 850 [ CSSD]2013-05-23 15:48:52.807 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 15:48:52.807 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095b30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:52.807 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:48:52.807 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 851 (0x7f431bfd9990) [ CSSD]2013-05-23 15:48:52.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9990 (RPC#851) tag(353002c) sent to node 1 [ CSSD]2013-05-23 15:48:52.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0353002c) incarnation(174631051) [ CSSD]2013-05-23 15:48:52.808 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:48:52.808 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9990 (RPC#851) state 6, flags 0x100 [ CSSD]2013-05-23 15:48:52.808 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9990, ret 0, client 0x7f431c095b30 member 0x7f431c136de0 [ CSSD]2013-05-23 15:48:52.808 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 851 [ CSSD]2013-05-23 15:48:52.808 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:48:53.011 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136f00 [ CSSD]2013-05-23 15:48:53.011 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136f00) con(0x7f431c091480) skgpid ospid 19400 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:48:53.406 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:53.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:53.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:55.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:55.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:55.672 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:55.773 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:48:55.773 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 852 (0x7f431bfd9a10) [ CSSD]2013-05-23 15:48:55.773 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9a10 (RPC#852) tag(354002c) sent to node 1 [ CSSD]2013-05-23 15:48:55.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:55.774 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 852 (0x7f431bfd9a10) [ CSSD]2013-05-23 15:48:55.774 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9a10 (RPC#852) state 6, flags 0x202 [ CSSD]2013-05-23 15:48:55.774 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9a10, ret 0, client (nil) [ CSSD]2013-05-23 15:48:55.774 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 852 [ CSSD]2013-05-23 15:48:56.196 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:48:56.196 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:48:57.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:57.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:57.678 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:48:59.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:48:59.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:48:59.730 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:01.199 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:01.200 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:01.515 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:01.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:01.756 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:03.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:03.347 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014157, gin 1, member 0 [ CSSD]2013-05-23 15:49:03.347 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:49:03.347 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:49:03.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:03.353 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:49:03.353 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f83f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:49:03.526 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:03.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:03.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:05.531 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:05.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:05.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:05.808 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:49:05.809 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 853 (0x7f431bfd9a90) [ CSSD]2013-05-23 15:49:05.809 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9a90 (RPC#853) tag(355002c) sent to node 1 [ CSSD]2013-05-23 15:49:05.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:05.809 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 853 (0x7f431bfd9a90) [ CSSD]2013-05-23 15:49:05.809 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9a90 (RPC#853) state 6, flags 0x202 [ CSSD]2013-05-23 15:49:05.809 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9a90, ret 0, client (nil) [ CSSD]2013-05-23 15:49:05.809 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 853 [ CSSD]2013-05-23 15:49:06.202 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:06.202 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:07.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:07.219 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014158, gin 1, member 0 [ CSSD]2013-05-23 15:49:07.219 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:49:07.219 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:49:07.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:07.225 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:49:07.225 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2323300) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:49:07.583 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:07.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:07.779 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:08.053 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136f00) allocated [ CSSD]2013-05-23 15:49:08.054 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:49:08.055 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136f00 - 1,2,3 [ CSSD]2013-05-23 15:49:08.055 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090ef0) proc(0x7f431c136f00) pid(19574) version 11:1:1:4 [ CSSD]2013-05-23 15:49:08.060 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c136f00), client(1/0x7f431c13f730) [ CSSD]2013-05-23 15:49:08.060 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:49:08.060 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c137160, requested num -1 [ CSSD]2013-05-23 15:49:08.060 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:08.060 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 854 (0x7f431bfd9b10) [ CSSD]2013-05-23 15:49:08.060 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9b10 (RPC#854) tag(356002c) sent to node 1 [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0356002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014159, gin 1, member 0 [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137740/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9b10 (RPC#854) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9b10, ret 0, client 0x7f431c13f730 member 0x7f431c136de0 [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 15:49:08.061 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 854 [ CSSD]2013-05-23 15:49:08.067 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:49:08.067 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:08.067 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:08.067 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 855 (0x7f431bfd9b90) [ CSSD]2013-05-23 15:49:08.068 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9b90 (RPC#855) tag(357002c) sent to node 1 [ CSSD]2013-05-23 15:49:08.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0357002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:08.068 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:08.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9b90 (RPC#855) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:08.068 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9b90, ret 0, client 0x7f431c13f730 member 0x7f431c136de0 [ CSSD]2013-05-23 15:49:08.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 855 [ CSSD]2013-05-23 15:49:08.068 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:49:08.271 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136f00 [ CSSD]2013-05-23 15:49:08.271 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136f00) con(0x7f431c090ef0) skgpid ospid 19574 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:49:09.589 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:09.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:09.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:10.972 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136f00) allocated [ CSSD]2013-05-23 15:49:10.973 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:49:10.974 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136f00 - 1,2,3 [ CSSD]2013-05-23 15:49:10.974 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090ef0) proc(0x7f431c136f00) pid(19657) version 11:1:1:4 [ CSSD]2013-05-23 15:49:10.980 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c136f00), client(1/0x7f431c095b30) [ CSSD]2013-05-23 15:49:10.980 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 15:49:10.980 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095b30 with con 0x7f431c137160, requested num -1 [ CSSD]2013-05-23 15:49:10.980 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:10.980 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 856 (0x7f431bfd9c10) [ CSSD]2013-05-23 15:49:10.980 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9c10 (RPC#856) tag(358002c) sent to node 1 [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0358002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014160, gin 1, member 0 [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137740/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9c10 (RPC#856) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9c10, ret 0, client 0x7f431c095b30 member 0x7f431c136de0 [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095b30 [ CSSD]2013-05-23 15:49:10.981 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 856 [ CSSD]2013-05-23 15:49:10.987 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 15:49:10.987 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095b30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:10.987 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:10.987 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 857 (0x7f431bfd9c90) [ CSSD]2013-05-23 15:49:10.987 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9c90 (RPC#857) tag(359002c) sent to node 1 [ CSSD]2013-05-23 15:49:10.987 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0359002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:10.987 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:10.987 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9c90 (RPC#857) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:10.987 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9c90, ret 0, client 0x7f431c095b30 member 0x7f431c136de0 [ CSSD]2013-05-23 15:49:10.988 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 857 [ CSSD]2013-05-23 15:49:10.988 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:49:11.190 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136f00 [ CSSD]2013-05-23 15:49:11.190 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136f00) con(0x7f431c090ef0) skgpid ospid 19657 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:49:11.205 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:11.206 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:11.595 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:11.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:11.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:13.686 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:13.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:13.807 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:15.208 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:15.208 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:49:15.692 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:15.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:15.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:15.835 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:49:15.835 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 858 (0x7f431bfd9d10) [ CSSD]2013-05-23 15:49:15.835 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9d10 (RPC#858) tag(35a002c) sent to node 1 [ CSSD]2013-05-23 15:49:15.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:15.836 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 858 (0x7f431bfd9d10) [ CSSD]2013-05-23 15:49:15.836 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9d10 (RPC#858) state 6, flags 0x202 [ CSSD]2013-05-23 15:49:15.836 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9d10, ret 0, client (nil) [ CSSD]2013-05-23 15:49:15.836 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 858 [ CSSD]2013-05-23 15:49:17.704 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:17.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:17.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:18.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:18.609 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014161, gin 1, member 0 [ CSSD]2013-05-23 15:49:18.609 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:49:18.609 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:49:18.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:18.616 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:49:18.616 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f83f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:49:19.710 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:19.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:19.891 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:20.211 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:20.211 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:21.715 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:21.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:21.897 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:23.209 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:49:23.209 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:49:23.211 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:49:23.211 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:49:23.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:23.292 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014162, gin 1, member 0 [ CSSD]2013-05-23 15:49:23.292 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f80f0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 15:49:23.292 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 15:49:23.298 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:23.298 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 15:49:23.298 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x2323300) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:49:23.301 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136f00) allocated [ CSSD]2013-05-23 15:49:23.302 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:49:23.302 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136f00 - 1,2,3 [ CSSD]2013-05-23 15:49:23.302 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091480) proc(0x7f431c136f00) pid(19747) version 11:1:1:4 [ CSSD]2013-05-23 15:49:23.308 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c136f00), client(1/0x22f7e90) [ CSSD]2013-05-23 15:49:23.308 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f7e90) [ CSSD]2013-05-23 15:49:23.308 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f7e90 with con 0x22f8270, requested num -1 [ CSSD]2013-05-23 15:49:23.308 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:23.308 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 859 (0x7f431bfd9d90) [ CSSD]2013-05-23 15:49:23.308 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9d90 (RPC#859) tag(35b002c) sent to node 1 [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(035b002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014163, gin 1, member 0 [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23235e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9d90 (RPC#859) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9d90, ret 0, client 0x22f7e90 member 0x23238e0 [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f7e90 [ CSSD]2013-05-23 15:49:23.309 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 859 [ CSSD]2013-05-23 15:49:23.315 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f7e90) [ CSSD]2013-05-23 15:49:23.315 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f7e90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:23.315 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23238e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:23.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 860 (0x7f431bfd9e10) [ CSSD]2013-05-23 15:49:23.315 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9e10 (RPC#860) tag(35c002c) sent to node 1 [ CSSD]2013-05-23 15:49:23.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(035c002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:23.315 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:23.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9e10 (RPC#860) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:23.315 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9e10, ret 0, client 0x22f7e90 member 0x23238e0 [ CSSD]2013-05-23 15:49:23.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 860 [ CSSD]2013-05-23 15:49:23.316 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23238e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:49:23.518 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136f00 [ CSSD]2013-05-23 15:49:23.518 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136f00) con(0x7f431c091480) skgpid ospid 19747 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:49:23.726 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:23.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:23.949 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:25.214 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:25.214 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:25.731 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:25.864 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:49:25.865 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 861 (0x7f431bfd9e90) [ CSSD]2013-05-23 15:49:25.865 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9e90 (RPC#861) tag(35d002c) sent to node 1 [ CSSD]2013-05-23 15:49:25.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:25.865 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 861 (0x7f431bfd9e90) [ CSSD]2013-05-23 15:49:25.865 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9e90 (RPC#861) state 6, flags 0x202 [ CSSD]2013-05-23 15:49:25.865 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9e90, ret 0, client (nil) [ CSSD]2013-05-23 15:49:25.865 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 861 [ CSSD]2013-05-23 15:49:25.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:25.955 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:27.784 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:27.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:27.962 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:29.790 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:29.972 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:29.972 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:30.217 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:30.217 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:31.795 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:31.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:31.978 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:33.807 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:33.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:33.859 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014164, gin 1, member 0 [ CSSD]2013-05-23 15:49:33.859 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:49:33.859 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:49:33.866 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:33.866 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:49:33.866 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f83f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:49:33.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:33.988 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:35.220 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:35.221 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:35.812 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:35.889 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:49:35.889 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 862 (0x7f431bfd9f10) [ CSSD]2013-05-23 15:49:35.890 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9f10 (RPC#862) tag(35e002c) sent to node 1 [ CSSD]2013-05-23 15:49:35.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:35.890 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 862 (0x7f431bfd9f10) [ CSSD]2013-05-23 15:49:35.890 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9f10 (RPC#862) state 6, flags 0x202 [ CSSD]2013-05-23 15:49:35.890 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9f10, ret 0, client (nil) [ CSSD]2013-05-23 15:49:35.890 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 862 [ CSSD]2013-05-23 15:49:36.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:36.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:37.132 [1191196992] >TRACE: clssgmAllocProc: (0x2324180) allocated [ CSSD]2013-05-23 15:49:37.133 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:49:37.134 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2324180 - 1,2,3 [ CSSD]2013-05-23 15:49:37.134 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23233c0) proc(0x2324180) pid(19451/19451) version 11:1:1:4 [ CSSD]2013-05-23 15:49:37.139 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2324180), client(1/0x231ee10) [ CSSD]2013-05-23 15:49:37.139 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x231ee10) [ CSSD]2013-05-23 15:49:37.139 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:49:37.139 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 15:49:37.905 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:38.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:38.019 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:38.550 [1191196992] >TRACE: clssgmAllocProc: (0x22f2f60) allocated [ CSSD]2013-05-23 15:49:38.551 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:49:38.551 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2f60 - 1,2,3 [ CSSD]2013-05-23 15:49:38.551 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f17f0) proc(0x22f2f60) pid(19897) version 11:1:1:4 [ CSSD]2013-05-23 15:49:38.557 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22f2f60), client(1/0x22ede70) [ CSSD]2013-05-23 15:49:38.557 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ede70) [ CSSD]2013-05-23 15:49:38.557 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ede70 with con 0x22ed960, requested num -1 [ CSSD]2013-05-23 15:49:38.557 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:38.557 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 863 (0x7f431bfd9f90) [ CSSD]2013-05-23 15:49:38.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9f90 (RPC#863) tag(35f002c) sent to node 1 [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(035f002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014165, gin 1, member 0 [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9f90 (RPC#863) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9f90, ret 0, client 0x22ede70 member 0x22f2e40 [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ede70 [ CSSD]2013-05-23 15:49:38.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 863 [ CSSD]2013-05-23 15:49:38.564 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ede70) [ CSSD]2013-05-23 15:49:38.564 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ede70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:38.564 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:38.564 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 864 (0x7f431bfda010) [ CSSD]2013-05-23 15:49:38.564 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda010 (RPC#864) tag(360002c) sent to node 1 [ CSSD]2013-05-23 15:49:38.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0360002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:38.564 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:38.564 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda010 (RPC#864) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:38.564 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda010, ret 0, client 0x22ede70 member 0x22f2e40 [ CSSD]2013-05-23 15:49:38.565 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 864 [ CSSD]2013-05-23 15:49:38.565 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:49:38.767 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2f60 [ CSSD]2013-05-23 15:49:38.767 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2f60) con(0x22f17f0) skgpid ospid 19897 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:49:39.223 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:39.223 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:49:39.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:40.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:40.071 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:41.707 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2324180), client(2/0x22f1e60) [ CSSD]2013-05-23 15:49:41.707 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f1e60) [ CSSD]2013-05-23 15:49:41.707 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:49:41.707 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 15:49:41.916 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:42.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:42.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:43.927 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:44.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:44.090 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:44.226 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:44.227 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:45.916 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:49:45.917 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 865 (0x7f431bfda090) [ CSSD]2013-05-23 15:49:45.917 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda090 (RPC#865) tag(361002c) sent to node 1 [ CSSD]2013-05-23 15:49:45.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:45.917 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 865 (0x7f431bfda090) [ CSSD]2013-05-23 15:49:45.917 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda090 (RPC#865) state 6, flags 0x202 [ CSSD]2013-05-23 15:49:45.917 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda090, ret 0, client (nil) [ CSSD]2013-05-23 15:49:45.917 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 865 [ CSSD]2013-05-23 15:49:45.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:46.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:46.095 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:47.985 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:48.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:48.101 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:49.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:49.106 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014166, gin 1, member 0 [ CSSD]2013-05-23 15:49:49.106 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:49:49.106 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:49:49.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:49.112 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:49:49.112 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:49:49.229 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:49.230 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:49.991 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:50.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:50.152 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:51.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:52.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:52.178 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:53.799 [1191196992] >TRACE: clssgmAllocProc: (0x22edb70) allocated [ CSSD]2013-05-23 15:49:53.800 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:49:53.801 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edb70 - 1,2,3 [ CSSD]2013-05-23 15:49:53.801 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1fd0) proc(0x22edb70) pid(20048) version 11:1:1:4 [ CSSD]2013-05-23 15:49:53.817 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22edb70), client(1/0x22ee0d0) [ CSSD]2013-05-23 15:49:53.818 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee0d0) [ CSSD]2013-05-23 15:49:53.818 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee0d0 with con 0x22eddd0, requested num -1 [ CSSD]2013-05-23 15:49:53.818 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:53.818 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 866 (0x7f431bfda110) [ CSSD]2013-05-23 15:49:53.818 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda110 (RPC#866) tag(362002c) sent to node 1 [ CSSD]2013-05-23 15:49:53.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0362002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:53.818 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014167, gin 1, member 0 [ CSSD]2013-05-23 15:49:53.818 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:49:53.819 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:49:53.819 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:49:53.819 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda110 (RPC#866) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:53.819 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda110, ret 0, client 0x22ee0d0 member 0x22eda50 [ CSSD]2013-05-23 15:49:53.819 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee0d0 [ CSSD]2013-05-23 15:49:53.819 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 866 [ CSSD]2013-05-23 15:49:53.825 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee0d0) [ CSSD]2013-05-23 15:49:53.825 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee0d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:53.825 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eda50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:49:53.825 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 867 (0x7f431bfda190) [ CSSD]2013-05-23 15:49:53.825 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda190 (RPC#867) tag(363002c) sent to node 1 [ CSSD]2013-05-23 15:49:53.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0363002c) incarnation(174631051) [ CSSD]2013-05-23 15:49:53.826 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:49:53.826 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda190 (RPC#867) state 6, flags 0x100 [ CSSD]2013-05-23 15:49:53.826 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda190, ret 0, client 0x22ee0d0 member 0x22eda50 [ CSSD]2013-05-23 15:49:53.826 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 867 [ CSSD]2013-05-23 15:49:53.826 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eda50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:49:54.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:54.028 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edb70 [ CSSD]2013-05-23 15:49:54.028 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edb70) con(0x22f1fd0) skgpid ospid 20048 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:49:54.233 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:54.233 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:49:54.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:54.270 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:55.939 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:49:55.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 868 (0x7f431bfda210) [ CSSD]2013-05-23 15:49:55.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda210 (RPC#868) tag(364002c) sent to node 1 [ CSSD]2013-05-23 15:49:55.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:55.940 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 868 (0x7f431bfda210) [ CSSD]2013-05-23 15:49:55.940 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda210 (RPC#868) state 6, flags 0x202 [ CSSD]2013-05-23 15:49:55.940 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda210, ret 0, client (nil) [ CSSD]2013-05-23 15:49:55.940 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 868 [ CSSD]2013-05-23 15:49:56.013 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:56.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:56.296 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:49:58.025 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:49:58.235 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:49:58.235 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:49:58.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:49:58.301 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:00.031 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:00.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:00.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:02.037 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:02.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:02.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:03.239 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:03.239 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:04.078 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:04.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:04.356 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014168, gin 1, member 0 [ CSSD]2013-05-23 15:50:04.356 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:50:04.356 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:50:04.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:04.363 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:50:04.363 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f23c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:50:04.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:04.371 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:05.963 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:50:05.964 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 869 (0x7f431bfda290) [ CSSD]2013-05-23 15:50:05.964 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda290 (RPC#869) tag(365002c) sent to node 1 [ CSSD]2013-05-23 15:50:05.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:05.964 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 869 (0x7f431bfda290) [ CSSD]2013-05-23 15:50:05.964 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda290 (RPC#869) state 6, flags 0x202 [ CSSD]2013-05-23 15:50:05.964 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda290, ret 0, client (nil) [ CSSD]2013-05-23 15:50:05.965 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 869 [ CSSD]2013-05-23 15:50:06.083 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:06.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:06.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:07.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:07.486 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014169, gin 1, member 0 [ CSSD]2013-05-23 15:50:07.486 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:50:07.486 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:50:07.493 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:07.493 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:50:07.493 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:50:08.096 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:08.099 [1191196992] >TRACE: clssgmAllocProc: (0x22edd50) allocated [ CSSD]2013-05-23 15:50:08.100 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:50:08.101 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edd50 - 1,2,3 [ CSSD]2013-05-23 15:50:08.101 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2680) proc(0x22edd50) pid(20383) version 11:1:1:4 [ CSSD]2013-05-23 15:50:08.242 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:08.242 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:08.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:08.381 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:08.606 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edd50 [ CSSD]2013-05-23 15:50:08.606 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edd50) con(0x22f2680) skgpid ospid 20383 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:09.061 [1191196992] >TRACE: clssgmAllocProc: (0x22edc40) allocated [ CSSD]2013-05-23 15:50:09.063 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:50:09.063 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edc40 - 1,2,3 [ CSSD]2013-05-23 15:50:09.063 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1c50) proc(0x22edc40) pid(20484) version 11:1:1:4 [ CSSD]2013-05-23 15:50:09.069 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22edc40), client(1/0x22ee1a0) [ CSSD]2013-05-23 15:50:09.069 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee1a0) [ CSSD]2013-05-23 15:50:09.069 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee1a0 with con 0x22edea0, requested num -1 [ CSSD]2013-05-23 15:50:09.069 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:09.069 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 870 (0x7f431bfda310) [ CSSD]2013-05-23 15:50:09.069 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda310 (RPC#870) tag(366002c) sent to node 1 [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0366002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014170, gin 1, member 0 [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda310 (RPC#870) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda310, ret 0, client 0x22ee1a0 member 0x22edb20 [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee1a0 [ CSSD]2013-05-23 15:50:09.070 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 870 [ CSSD]2013-05-23 15:50:09.076 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee1a0) [ CSSD]2013-05-23 15:50:09.076 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee1a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:09.076 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edb20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:09.076 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 871 (0x7f431bfda390) [ CSSD]2013-05-23 15:50:09.076 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda390 (RPC#871) tag(367002c) sent to node 1 [ CSSD]2013-05-23 15:50:09.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0367002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:09.076 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:09.076 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda390 (RPC#871) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:09.076 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda390, ret 0, client 0x22ee1a0 member 0x22edb20 [ CSSD]2013-05-23 15:50:09.077 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 871 [ CSSD]2013-05-23 15:50:09.077 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22edb20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:50:09.279 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edc40 [ CSSD]2013-05-23 15:50:09.279 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edc40) con(0x22f1c50) skgpid ospid 20484 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:10.101 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:10.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:10.393 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:11.230 [1191196992] >TRACE: clssgmAllocProc: (0x22f1d20) allocated [ CSSD]2013-05-23 15:50:11.231 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:50:11.231 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f1d20 - 1,2,3 [ CSSD]2013-05-23 15:50:11.231 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1fd0) proc(0x22f1d20) pid(20538) version 11:1:1:4 [ CSSD]2013-05-23 15:50:11.249 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22f1d20), client(1/0x22f3140) [ CSSD]2013-05-23 15:50:11.249 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3140) [ CSSD]2013-05-23 15:50:11.249 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3140 with con 0x22ee000, requested num -1 [ CSSD]2013-05-23 15:50:11.249 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:11.249 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 872 (0x7f431bfda410) [ CSSD]2013-05-23 15:50:11.249 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda410 (RPC#872) tag(368002c) sent to node 1 [ CSSD]2013-05-23 15:50:11.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0368002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:11.249 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014171, gin 1, member 0 [ CSSD]2013-05-23 15:50:11.249 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:50:11.250 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:50:11.250 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:50:11.250 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda410 (RPC#872) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:11.250 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda410, ret 0, client 0x22f3140 member 0x22ede80 [ CSSD]2013-05-23 15:50:11.250 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3140 [ CSSD]2013-05-23 15:50:11.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 872 [ CSSD]2013-05-23 15:50:11.256 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3140) [ CSSD]2013-05-23 15:50:11.256 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:11.256 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ede80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:11.256 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 873 (0x7f431bfda490) [ CSSD]2013-05-23 15:50:11.256 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda490 (RPC#873) tag(369002c) sent to node 1 [ CSSD]2013-05-23 15:50:11.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0369002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:11.256 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:11.256 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda490 (RPC#873) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:11.256 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda490, ret 0, client 0x22f3140 member 0x22ede80 [ CSSD]2013-05-23 15:50:11.257 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 873 [ CSSD]2013-05-23 15:50:11.257 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ede80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:50:11.459 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f1d20 [ CSSD]2013-05-23 15:50:11.459 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f1d20) con(0x22f1fd0) skgpid ospid 20538 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:12.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:12.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:12.417 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:13.245 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:13.245 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:14.120 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:14.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:14.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:15.988 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:50:15.988 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 874 (0x7f431bfda510) [ CSSD]2013-05-23 15:50:15.988 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda510 (RPC#874) tag(36a002c) sent to node 1 [ CSSD]2013-05-23 15:50:15.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:15.989 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 874 (0x7f431bfda510) [ CSSD]2013-05-23 15:50:15.989 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda510 (RPC#874) state 6, flags 0x202 [ CSSD]2013-05-23 15:50:15.989 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda510, ret 0, client (nil) [ CSSD]2013-05-23 15:50:15.989 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 874 [ CSSD]2013-05-23 15:50:16.125 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:16.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:16.457 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:17.248 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:17.248 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:50:18.216 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:18.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:18.482 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:18.673 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 15:50:18.673 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:50:18.673 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:50:18.673 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 15:50:18.673 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 15:50:18.673 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0840c0) skgpid 15159 ospid 15159 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:18.697 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:50:18.697 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:50:18.697 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:50:18.697 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 15:50:18.702 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:50:18.702 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:50:18.702 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:50:18.702 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 15:50:18.702 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 15:50:18.702 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c11ab50) skgpid 15167 ospid 15167 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:19.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:19.605 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014172, gin 1, member 0 [ CSSD]2013-05-23 15:50:19.605 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:50:19.605 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:50:19.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:19.612 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:50:19.612 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:50:20.222 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:20.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:20.574 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:22.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:22.251 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:22.251 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:22.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:22.599 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:24.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:24.310 [1191196992] >TRACE: clssgmAllocProc: (0x22ede10) allocated [ CSSD]2013-05-23 15:50:24.311 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:50:24.312 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ede10 - 1,2,3 [ CSSD]2013-05-23 15:50:24.312 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2750) proc(0x22ede10) pid(20670) version 11:1:1:4 [ CSSD]2013-05-23 15:50:24.318 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ede10), client(1/0x2317b60) [ CSSD]2013-05-23 15:50:24.318 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317b60) [ CSSD]2013-05-23 15:50:24.318 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317b60 with con 0x22f3120, requested num -1 [ CSSD]2013-05-23 15:50:24.318 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:24.318 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 875 (0x7f431bfda590) [ CSSD]2013-05-23 15:50:24.318 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda590 (RPC#875) tag(36b002c) sent to node 1 [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(036b002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014173, gin 1, member 0 [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317cd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda590 (RPC#875) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda590, ret 0, client 0x2317b60 member 0x22ee160 [ CSSD]2013-05-23 15:50:24.318 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317b60 [ CSSD]2013-05-23 15:50:24.319 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 875 [ CSSD]2013-05-23 15:50:24.325 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317b60) [ CSSD]2013-05-23 15:50:24.325 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317b60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:24.325 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ee160) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:24.325 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 876 (0x7f431bfda610) [ CSSD]2013-05-23 15:50:24.325 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda610 (RPC#876) tag(36c002c) sent to node 1 [ CSSD]2013-05-23 15:50:24.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(036c002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:24.325 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:24.325 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda610 (RPC#876) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:24.325 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda610, ret 0, client 0x2317b60 member 0x22ee160 [ CSSD]2013-05-23 15:50:24.326 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 876 [ CSSD]2013-05-23 15:50:24.326 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ee160) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:50:24.528 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ede10 [ CSSD]2013-05-23 15:50:24.528 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ede10) con(0x22f2750) skgpid ospid 20670 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:24.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:24.671 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:26.013 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:50:26.013 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 877 (0x7f431bfda690) [ CSSD]2013-05-23 15:50:26.014 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda690 (RPC#877) tag(36d002c) sent to node 1 [ CSSD]2013-05-23 15:50:26.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:26.014 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 877 (0x7f431bfda690) [ CSSD]2013-05-23 15:50:26.014 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda690 (RPC#877) state 6, flags 0x202 [ CSSD]2013-05-23 15:50:26.014 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda690, ret 0, client (nil) [ CSSD]2013-05-23 15:50:26.014 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 877 [ CSSD]2013-05-23 15:50:26.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:26.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:26.677 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:27.254 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:27.255 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:28.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:28.690 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:28.690 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:30.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:30.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:30.695 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:32.258 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:32.258 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:32.315 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:32.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:32.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:34.320 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:34.741 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:34.741 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:34.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:34.854 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014174, gin 1, member 0 [ CSSD]2013-05-23 15:50:34.854 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:50:34.854 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:50:34.861 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:34.861 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:50:34.861 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:50:36.040 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:50:36.040 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 878 (0x7f431bfda710) [ CSSD]2013-05-23 15:50:36.040 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda710 (RPC#878) tag(36e002c) sent to node 1 [ CSSD]2013-05-23 15:50:36.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:36.041 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 878 (0x7f431bfda710) [ CSSD]2013-05-23 15:50:36.041 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda710 (RPC#878) state 6, flags 0x202 [ CSSD]2013-05-23 15:50:36.041 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda710, ret 0, client (nil) [ CSSD]2013-05-23 15:50:36.041 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 878 [ CSSD]2013-05-23 15:50:36.261 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:36.261 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:50:36.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:36.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:36.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:38.338 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:38.760 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:38.760 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:39.560 [1191196992] >TRACE: clssgmAllocProc: (0x22f1d20) allocated [ CSSD]2013-05-23 15:50:39.561 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:50:39.561 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f1d20 - 1,2,3 [ CSSD]2013-05-23 15:50:39.561 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2680) proc(0x22f1d20) pid(20827) version 11:1:1:4 [ CSSD]2013-05-23 15:50:39.577 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f1d20), client(1/0x22ee0e0) [ CSSD]2013-05-23 15:50:39.577 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee0e0) [ CSSD]2013-05-23 15:50:39.577 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee0e0 with con 0x22f3120, requested num -1 [ CSSD]2013-05-23 15:50:39.577 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:39.577 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 879 (0x7f431bfda790) [ CSSD]2013-05-23 15:50:39.577 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda790 (RPC#879) tag(36f002c) sent to node 1 [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(036f002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014175, gin 1, member 0 [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317aa0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda790 (RPC#879) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda790, ret 0, client 0x22ee0e0 member 0x22edc30 [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee0e0 [ CSSD]2013-05-23 15:50:39.578 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 879 [ CSSD]2013-05-23 15:50:39.584 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee0e0) [ CSSD]2013-05-23 15:50:39.584 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee0e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:39.584 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edc30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:39.584 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 880 (0x7f431bfda810) [ CSSD]2013-05-23 15:50:39.585 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda810 (RPC#880) tag(370002c) sent to node 1 [ CSSD]2013-05-23 15:50:39.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0370002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:39.585 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:39.585 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda810 (RPC#880) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:39.585 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda810, ret 0, client 0x22ee0e0 member 0x22edc30 [ CSSD]2013-05-23 15:50:39.585 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 880 [ CSSD]2013-05-23 15:50:39.585 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22edc30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:50:39.787 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f1d20 [ CSSD]2013-05-23 15:50:39.787 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f1d20) con(0x22f2680) skgpid ospid 20827 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:40.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:40.765 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:40.765 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:41.264 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:41.265 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:42.395 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:42.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:42.771 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:44.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:44.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:44.783 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:46.066 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:50:46.066 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 881 (0x7f431bfda890) [ CSSD]2013-05-23 15:50:46.066 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda890 (RPC#881) tag(371002c) sent to node 1 [ CSSD]2013-05-23 15:50:46.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:46.067 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 881 (0x7f431bfda890) [ CSSD]2013-05-23 15:50:46.067 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda890 (RPC#881) state 6, flags 0x202 [ CSSD]2013-05-23 15:50:46.067 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda890, ret 0, client (nil) [ CSSD]2013-05-23 15:50:46.067 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 881 [ CSSD]2013-05-23 15:50:46.267 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:46.268 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:46.406 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:46.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:46.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:48.417 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:48.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:48.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:50.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:50.104 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014176, gin 1, member 0 [ CSSD]2013-05-23 15:50:50.104 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:50:50.104 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:50:50.111 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:50.111 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:50:50.111 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:50:50.423 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:50.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:50.807 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:51.270 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:51.271 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:50:52.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:52.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:52.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:54.441 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:54.829 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 15:50:54.830 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:50:54.831 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 15:50:54.831 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c10a190) pid(20983) version 11:1:1:4 [ CSSD]2013-05-23 15:50:54.837 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a190), client(1/0x7f431c095b30) [ CSSD]2013-05-23 15:50:54.837 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 15:50:54.837 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095b30 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 15:50:54.837 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:54.837 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 882 (0x7f431bfda910) [ CSSD]2013-05-23 15:50:54.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda910 (RPC#882) tag(372002c) sent to node 1 [ CSSD]2013-05-23 15:50:54.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0372002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:54.837 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014177, gin 1, member 0 [ CSSD]2013-05-23 15:50:54.837 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:50:54.838 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:50:54.838 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:50:54.838 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda910 (RPC#882) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:54.838 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda910, ret 0, client 0x7f431c095b30 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:50:54.838 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095b30 [ CSSD]2013-05-23 15:50:54.838 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 882 [ CSSD]2013-05-23 15:50:54.843 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 15:50:54.843 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095b30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:54.843 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:54.843 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 883 (0x7f431bfda990) [ CSSD]2013-05-23 15:50:54.843 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda990 (RPC#883) tag(373002c) sent to node 1 [ CSSD]2013-05-23 15:50:54.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0373002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:54.844 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:54.844 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda990 (RPC#883) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:54.844 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda990, ret 0, client 0x7f431c095b30 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:50:54.844 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 883 [ CSSD]2013-05-23 15:50:54.844 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:50:54.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:54.884 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:55.046 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 15:50:55.046 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e5200) skgpid ospid 20983 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:55.273 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:50:55.273 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:50:55.607 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 15:50:55.608 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:50:55.609 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 15:50:55.609 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c10a190) pid(21030) version 11:1:1:4 [ CSSD]2013-05-23 15:50:55.615 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a190), client(1/0x7f431c13f730) [ CSSD]2013-05-23 15:50:55.615 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:50:55.615 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:50:55.615 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:55.615 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 884 (0x7f431bfdaa10) [ CSSD]2013-05-23 15:50:55.615 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaa10 (RPC#884) tag(374002c) sent to node 1 [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0374002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014178, gin 1, member 0 [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaa10 (RPC#884) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdaa10, ret 0, client 0x7f431c13f730 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 15:50:55.616 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 884 [ CSSD]2013-05-23 15:50:55.621 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 15:50:55.621 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:55.621 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:50:55.621 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 885 (0x7f431bfdaa90) [ CSSD]2013-05-23 15:50:55.622 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaa90 (RPC#885) tag(375002c) sent to node 1 [ CSSD]2013-05-23 15:50:55.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0375002c) incarnation(174631051) [ CSSD]2013-05-23 15:50:55.622 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:50:55.622 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaa90 (RPC#885) state 6, flags 0x100 [ CSSD]2013-05-23 15:50:55.622 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdaa90, ret 0, client 0x7f431c13f730 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:50:55.622 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 885 [ CSSD]2013-05-23 15:50:55.622 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:50:55.824 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 15:50:55.824 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c12a550) skgpid ospid 21030 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:50:56.090 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:50:56.090 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 886 (0x7f431bfdab10) [ CSSD]2013-05-23 15:50:56.091 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdab10 (RPC#886) tag(376002c) sent to node 1 [ CSSD]2013-05-23 15:50:56.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:56.091 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 886 (0x7f431bfdab10) [ CSSD]2013-05-23 15:50:56.091 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdab10 (RPC#886) state 6, flags 0x202 [ CSSD]2013-05-23 15:50:56.091 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdab10, ret 0, client (nil) [ CSSD]2013-05-23 15:50:56.091 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 886 [ CSSD]2013-05-23 15:50:56.527 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:56.870 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:56.870 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:50:58.538 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:50:58.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:50:58.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:00.276 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:00.276 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:00.544 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:00.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:00.886 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:02.596 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:02.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:02.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:04.602 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:04.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:04.964 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:05.279 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:05.279 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:05.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:05.356 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014179, gin 1, member 0 [ CSSD]2013-05-23 15:51:05.356 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f20a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:51:05.356 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:51:05.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:05.362 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:51:05.362 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:51:06.115 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:51:06.116 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 887 (0x7f431bfdab90) [ CSSD]2013-05-23 15:51:06.116 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdab90 (RPC#887) tag(377002c) sent to node 1 [ CSSD]2013-05-23 15:51:06.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:06.116 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 887 (0x7f431bfdab90) [ CSSD]2013-05-23 15:51:06.116 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdab90 (RPC#887) state 6, flags 0x202 [ CSSD]2013-05-23 15:51:06.116 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdab90, ret 0, client (nil) [ CSSD]2013-05-23 15:51:06.116 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 887 [ CSSD]2013-05-23 15:51:06.608 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:06.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:06.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:07.747 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:07.747 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014180, gin 1, member 0 [ CSSD]2013-05-23 15:51:07.747 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:51:07.747 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:51:07.753 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:07.753 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:51:07.753 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:51:08.619 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:09.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:09.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:10.077 [1191196992] >TRACE: clssgmAllocProc: (0x22edb60) allocated [ CSSD]2013-05-23 15:51:10.078 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:51:10.078 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edb60 - 1,2,3 [ CSSD]2013-05-23 15:51:10.078 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1fd0) proc(0x22edb60) pid(21133) version 11:1:1:4 [ CSSD]2013-05-23 15:51:10.083 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22edb60), client(1/0x22ee0c0) [ CSSD]2013-05-23 15:51:10.083 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee0c0) [ CSSD]2013-05-23 15:51:10.083 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee0c0 with con 0x22eddc0, requested num -1 [ CSSD]2013-05-23 15:51:10.083 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:10.083 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 888 (0x7f431bfdac10) [ CSSD]2013-05-23 15:51:10.083 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdac10 (RPC#888) tag(378002c) sent to node 1 [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0378002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014181, gin 1, member 0 [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317aa0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdac10 (RPC#888) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdac10, ret 0, client 0x22ee0c0 member 0x22eda30 [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee0c0 [ CSSD]2013-05-23 15:51:10.084 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 888 [ CSSD]2013-05-23 15:51:10.091 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee0c0) [ CSSD]2013-05-23 15:51:10.091 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee0c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:10.091 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eda30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:10.091 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 889 (0x7f431bfdac90) [ CSSD]2013-05-23 15:51:10.091 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdac90 (RPC#889) tag(379002c) sent to node 1 [ CSSD]2013-05-23 15:51:10.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0379002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:10.092 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:10.092 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdac90 (RPC#889) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:10.092 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdac90, ret 0, client 0x22ee0c0 member 0x22eda30 [ CSSD]2013-05-23 15:51:10.092 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 889 [ CSSD]2013-05-23 15:51:10.092 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eda30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:51:10.282 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:10.283 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:10.294 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edb60 [ CSSD]2013-05-23 15:51:10.294 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edb60) con(0x22f1fd0) skgpid ospid 21133 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:10.625 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:11.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:11.007 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:11.496 [1191196992] >TRACE: clssgmAllocProc: (0x22edb60) allocated [ CSSD]2013-05-23 15:51:11.497 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:51:11.498 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edb60 - 1,2,3 [ CSSD]2013-05-23 15:51:11.498 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1fd0) proc(0x22edb60) pid(21186) version 11:1:1:4 [ CSSD]2013-05-23 15:51:11.503 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22edb60), client(1/0x22ee0c0) [ CSSD]2013-05-23 15:51:11.503 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee0c0) [ CSSD]2013-05-23 15:51:11.503 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee0c0 with con 0x22eddc0, requested num -1 [ CSSD]2013-05-23 15:51:11.503 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:11.503 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 890 (0x7f431bfdad10) [ CSSD]2013-05-23 15:51:11.503 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdad10 (RPC#890) tag(37a002c) sent to node 1 [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(037a002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014182, gin 1, member 0 [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317aa0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdad10 (RPC#890) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdad10, ret 0, client 0x22ee0c0 member 0x22eda30 [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee0c0 [ CSSD]2013-05-23 15:51:11.504 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 890 [ CSSD]2013-05-23 15:51:11.511 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee0c0) [ CSSD]2013-05-23 15:51:11.511 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee0c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:11.511 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eda30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:11.511 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 891 (0x7f431bfdad90) [ CSSD]2013-05-23 15:51:11.511 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdad90 (RPC#891) tag(37b002c) sent to node 1 [ CSSD]2013-05-23 15:51:11.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(037b002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:11.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:11.511 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdad90 (RPC#891) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:11.511 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdad90, ret 0, client 0x22ee0c0 member 0x22eda30 [ CSSD]2013-05-23 15:51:11.511 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 891 [ CSSD]2013-05-23 15:51:11.511 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eda30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:51:11.714 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edb60 [ CSSD]2013-05-23 15:51:11.714 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edb60) con(0x22f1fd0) skgpid ospid 21186 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:12.717 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:13.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:13.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:14.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:15.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:15.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:15.286 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:15.286 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:15.385 [1191196992] >TRACE: clssgmAllocProc: (0x22edc30) allocated [ CSSD]2013-05-23 15:51:15.386 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:51:15.387 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edc30 - 1,2,3 [ CSSD]2013-05-23 15:51:15.387 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2610) proc(0x22edc30) pid(21220) version 11:1:1:4 [ CSSD]2013-05-23 15:51:15.404 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22edc30), client(1/0x2317aa0) [ CSSD]2013-05-23 15:51:15.404 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317aa0) [ CSSD]2013-05-23 15:51:15.404 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x2317aa0 with con 0x22edee0, requested num -1 [ CSSD]2013-05-23 15:51:15.404 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:51:15.404 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 892 (0x7f431bfdae10) [ CSSD]2013-05-23 15:51:15.404 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdae10 (RPC#892) tag(37c002c) sent to node 1 [ CSSD]2013-05-23 15:51:15.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(037c002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:15.404 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014183, gin 1, member 0 [ CSSD]2013-05-23 15:51:15.404 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317c10/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 15:51:15.404 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:51:15.405 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:51:15.405 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdae10 (RPC#892) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:15.405 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdae10, ret 0, client 0x2317aa0 member 0x22edb20 [ CSSD]2013-05-23 15:51:15.405 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317aa0 [ CSSD]2013-05-23 15:51:15.405 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 892 [ CSSD]2013-05-23 15:51:15.411 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317aa0) [ CSSD]2013-05-23 15:51:15.411 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317aa0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:51:15.411 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edb20) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:51:15.411 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 893 (0x7f431bfdae90) [ CSSD]2013-05-23 15:51:15.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdae90 (RPC#893) tag(37d002c) sent to node 1 [ CSSD]2013-05-23 15:51:15.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(037d002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:15.411 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:51:15.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdae90 (RPC#893) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:15.411 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdae90, ret 0, client 0x2317aa0 member 0x22edb20 [ CSSD]2013-05-23 15:51:15.412 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 893 [ CSSD]2013-05-23 15:51:15.412 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22edb20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:51:15.522 [1191196992] >TRACE: clssgmAllocProc: (0x2319370) allocated [ CSSD]2013-05-23 15:51:15.524 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:51:15.524 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2319370 - 1,2,3 [ CSSD]2013-05-23 15:51:15.524 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1fd0) proc(0x2319370) pid(21225) version 11:1:1:4 [ CSSD]2013-05-23 15:51:15.549 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:51:15.551 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:51:15.554 [1191196992] >TRACE: clssgmDeadProc: proc 0x2319370 [ CSSD]2013-05-23 15:51:15.554 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2319370) con(0x22f1fd0) skgpid ospid 21225 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:15.614 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edc30 [ CSSD]2013-05-23 15:51:15.614 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edc30) con(0x22f2610) skgpid ospid 21220 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:16.152 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:51:16.152 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 894 (0x7f431bfdaf10) [ CSSD]2013-05-23 15:51:16.152 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaf10 (RPC#894) tag(37e002c) sent to node 1 [ CSSD]2013-05-23 15:51:16.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:16.153 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 894 (0x7f431bfdaf10) [ CSSD]2013-05-23 15:51:16.153 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaf10 (RPC#894) state 6, flags 0x202 [ CSSD]2013-05-23 15:51:16.153 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdaf10, ret 0, client (nil) [ CSSD]2013-05-23 15:51:16.153 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 894 [ CSSD]2013-05-23 15:51:16.728 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:17.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:17.070 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:18.748 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:19.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:19.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:20.289 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:20.290 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:20.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:20.606 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014184, gin 1, member 0 [ CSSD]2013-05-23 15:51:20.606 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f20a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:51:20.606 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:51:20.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:20.612 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:51:20.612 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:51:20.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:21.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:21.088 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:22.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:23.043 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:51:23.043 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:51:23.043 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:51:23.043 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 15:51:23.043 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 15:51:23.043 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c0e6720) skgpid 15782 ospid 15782 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:23.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:23.093 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:24.760 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:25.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:25.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:25.292 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:25.292 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:25.326 [1191196992] >TRACE: clssgmAllocProc: (0x22edd40) allocated [ CSSD]2013-05-23 15:51:25.329 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:51:25.330 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22edd40 - 1,2,3 [ CSSD]2013-05-23 15:51:25.330 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2660) proc(0x22edd40) pid(21334) version 11:1:1:4 [ CSSD]2013-05-23 15:51:25.346 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22edd40), client(1/0x2317aa0) [ CSSD]2013-05-23 15:51:25.346 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317aa0) [ CSSD]2013-05-23 15:51:25.346 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317aa0 with con 0x22f3100, requested num -1 [ CSSD]2013-05-23 15:51:25.346 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:25.346 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 895 (0x7f431bfdaf90) [ CSSD]2013-05-23 15:51:25.346 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaf90 (RPC#895) tag(37f002c) sent to node 1 [ CSSD]2013-05-23 15:51:25.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(037f002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:25.346 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014185, gin 1, member 0 [ CSSD]2013-05-23 15:51:25.346 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:51:25.347 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:51:25.347 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:51:25.347 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaf90 (RPC#895) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:25.347 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdaf90, ret 0, client 0x2317aa0 member 0x22edc30 [ CSSD]2013-05-23 15:51:25.347 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317aa0 [ CSSD]2013-05-23 15:51:25.347 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 895 [ CSSD]2013-05-23 15:51:25.352 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317aa0) [ CSSD]2013-05-23 15:51:25.352 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317aa0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:25.352 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edc30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:25.353 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 896 (0x7f431bfdb010) [ CSSD]2013-05-23 15:51:25.353 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb010 (RPC#896) tag(380002c) sent to node 1 [ CSSD]2013-05-23 15:51:25.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0380002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:25.353 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:25.353 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb010 (RPC#896) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:25.353 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb010, ret 0, client 0x2317aa0 member 0x22edc30 [ CSSD]2013-05-23 15:51:25.353 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 896 [ CSSD]2013-05-23 15:51:25.353 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22edc30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:51:25.556 [1191196992] >TRACE: clssgmDeadProc: proc 0x22edd40 [ CSSD]2013-05-23 15:51:25.556 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22edd40) con(0x22f2660) skgpid ospid 21334 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:26.197 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:51:26.197 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 897 (0x7f431bfdb090) [ CSSD]2013-05-23 15:51:26.198 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb090 (RPC#897) tag(381002c) sent to node 1 [ CSSD]2013-05-23 15:51:26.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:26.198 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 897 (0x7f431bfdb090) [ CSSD]2013-05-23 15:51:26.198 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb090 (RPC#897) state 6, flags 0x202 [ CSSD]2013-05-23 15:51:26.198 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb090, ret 0, client (nil) [ CSSD]2013-05-23 15:51:26.198 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 897 [ CSSD]2013-05-23 15:51:26.767 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:27.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:27.110 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:28.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:29.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:29.162 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:30.295 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:30.295 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:30.825 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:31.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:31.168 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:32.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:33.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:33.192 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:34.844 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:35.205 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:35.205 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:35.298 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:35.298 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:35.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:35.864 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014186, gin 1, member 0 [ CSSD]2013-05-23 15:51:35.864 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:51:35.864 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:51:35.870 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:35.870 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:51:35.870 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:51:36.224 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:51:36.225 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 898 (0x7f431bfdb110) [ CSSD]2013-05-23 15:51:36.225 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb110 (RPC#898) tag(382002c) sent to node 1 [ CSSD]2013-05-23 15:51:36.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:36.225 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 898 (0x7f431bfdb110) [ CSSD]2013-05-23 15:51:36.225 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb110 (RPC#898) state 6, flags 0x202 [ CSSD]2013-05-23 15:51:36.225 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb110, ret 0, client (nil) [ CSSD]2013-05-23 15:51:36.225 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 898 [ CSSD]2013-05-23 15:51:36.849 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:37.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:37.210 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:38.860 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:39.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:39.262 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:39.300 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:39.300 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:51:40.598 [1191196992] >TRACE: clssgmAllocProc: (0x22f1d20) allocated [ CSSD]2013-05-23 15:51:40.599 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:51:40.599 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f1d20 - 1,2,3 [ CSSD]2013-05-23 15:51:40.599 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3040) proc(0x22f1d20) pid(21487) version 11:1:1:4 [ CSSD]2013-05-23 15:51:40.605 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f1d20), client(1/0x22f28f0) [ CSSD]2013-05-23 15:51:40.605 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f28f0) [ CSSD]2013-05-23 15:51:40.605 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f28f0 with con 0x22ede70, requested num -1 [ CSSD]2013-05-23 15:51:40.605 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:40.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 899 (0x7f431bfdb190) [ CSSD]2013-05-23 15:51:40.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb190 (RPC#899) tag(383002c) sent to node 1 [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0383002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014187, gin 1, member 0 [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317aa0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb190 (RPC#899) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb190, ret 0, client 0x22f28f0 member 0x22edc30 [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f28f0 [ CSSD]2013-05-23 15:51:40.606 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 899 [ CSSD]2013-05-23 15:51:40.612 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f28f0) [ CSSD]2013-05-23 15:51:40.612 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f28f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:40.612 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22edc30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:40.612 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 900 (0x7f431bfdb210) [ CSSD]2013-05-23 15:51:40.612 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb210 (RPC#900) tag(384002c) sent to node 1 [ CSSD]2013-05-23 15:51:40.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0384002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:40.613 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:40.613 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb210 (RPC#900) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:40.613 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb210, ret 0, client 0x22f28f0 member 0x22edc30 [ CSSD]2013-05-23 15:51:40.613 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 900 [ CSSD]2013-05-23 15:51:40.613 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22edc30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:51:40.815 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f1d20 [ CSSD]2013-05-23 15:51:40.815 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f1d20) con(0x22f3040) skgpid ospid 21487 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:40.866 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:41.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:41.268 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:42.918 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:43.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:43.274 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:44.303 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:44.304 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:44.924 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:45.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:45.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:46.249 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:51:46.249 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 901 (0x7f431bfdb290) [ CSSD]2013-05-23 15:51:46.250 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb290 (RPC#901) tag(385002c) sent to node 1 [ CSSD]2013-05-23 15:51:46.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:46.250 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 901 (0x7f431bfdb290) [ CSSD]2013-05-23 15:51:46.250 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb290 (RPC#901) state 6, flags 0x202 [ CSSD]2013-05-23 15:51:46.250 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb290, ret 0, client (nil) [ CSSD]2013-05-23 15:51:46.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 901 [ CSSD]2013-05-23 15:51:46.929 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:47.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:47.291 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:48.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:49.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:49.282 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:49.307 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:49.307 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:51.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:51.114 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:51.114 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014188, gin 1, member 0 [ CSSD]2013-05-23 15:51:51.114 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:51:51.114 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:51:51.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:51.120 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:51:51.120 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:51:51.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:51.307 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:53.019 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:53.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:53.313 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:54.310 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:54.310 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:55.025 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:55.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:55.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:55.847 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:51:55.848 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:51:55.849 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:51:55.849 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c135290) pid(21653) version 11:1:1:4 [ CSSD]2013-05-23 15:51:55.854 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:51:55.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:51:55.855 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:51:55.855 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:55.855 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 902 (0x7f431bfdb310) [ CSSD]2013-05-23 15:51:55.855 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb310 (RPC#902) tag(386002c) sent to node 1 [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0386002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014189, gin 1, member 0 [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb310 (RPC#902) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb310, ret 0, client 0x7f431c12e6d0 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:51:55.855 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:51:55.856 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 902 [ CSSD]2013-05-23 15:51:55.862 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:51:55.862 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:55.862 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:51:55.862 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 903 (0x7f431bfdb390) [ CSSD]2013-05-23 15:51:55.862 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb390 (RPC#903) tag(387002c) sent to node 1 [ CSSD]2013-05-23 15:51:55.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0387002c) incarnation(174631051) [ CSSD]2013-05-23 15:51:55.862 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:51:55.862 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb390 (RPC#903) state 6, flags 0x100 [ CSSD]2013-05-23 15:51:55.862 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb390, ret 0, client 0x7f431c12e6d0 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:51:55.862 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 903 [ CSSD]2013-05-23 15:51:55.862 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:51:56.065 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:51:56.065 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c12a550) skgpid ospid 21653 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:51:56.274 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:51:56.274 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 904 (0x7f431bfdb410) [ CSSD]2013-05-23 15:51:56.274 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb410 (RPC#904) tag(388002c) sent to node 1 [ CSSD]2013-05-23 15:51:56.275 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:56.275 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 904 (0x7f431bfdb410) [ CSSD]2013-05-23 15:51:56.275 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb410 (RPC#904) state 6, flags 0x202 [ CSSD]2013-05-23 15:51:56.275 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb410, ret 0, client (nil) [ CSSD]2013-05-23 15:51:56.275 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 904 [ CSSD]2013-05-23 15:51:57.031 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:57.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:57.322 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:51:59.041 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:51:59.313 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:51:59.313 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:51:59.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:51:59.374 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:01.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:01.380 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:01.380 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:03.139 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:03.386 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:03.386 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:04.316 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:04.316 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:05.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:05.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:05.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:05.636 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:05.638 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:52:05.638 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:05.638 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c135290) pid(21761) version 11:1:1:4 [ CSSD]2013-05-23 15:52:05.644 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:05.644 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:05.644 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:52:05.644 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:05.644 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 905 (0x7f431bfdb490) [ CSSD]2013-05-23 15:52:05.644 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb490 (RPC#905) tag(389002c) sent to node 1 [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0389002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014190, gin 1, member 0 [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb490 (RPC#905) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb490, ret 0, client 0x7f431c12e6d0 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:52:05.645 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 905 [ CSSD]2013-05-23 15:52:05.651 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:05.651 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:05.651 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:05.651 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 906 (0x7f431bfdb510) [ CSSD]2013-05-23 15:52:05.651 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb510 (RPC#906) tag(38a002c) sent to node 1 [ CSSD]2013-05-23 15:52:05.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(038a002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:05.651 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:05.652 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb510 (RPC#906) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:05.652 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb510, ret 0, client 0x7f431c12e6d0 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:52:05.652 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 906 [ CSSD]2013-05-23 15:52:05.652 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:05.854 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:05.854 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c0840c0) skgpid ospid 21761 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:06.301 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:52:06.302 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 907 (0x7f431bfdb590) [ CSSD]2013-05-23 15:52:06.302 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb590 (RPC#907) tag(38b002c) sent to node 1 [ CSSD]2013-05-23 15:52:06.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:06.303 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 907 (0x7f431bfdb590) [ CSSD]2013-05-23 15:52:06.303 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb590 (RPC#907) state 6, flags 0x202 [ CSSD]2013-05-23 15:52:06.303 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb590, ret 0, client (nil) [ CSSD]2013-05-23 15:52:06.303 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 907 [ CSSD]2013-05-23 15:52:06.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:06.363 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014191, gin 1, member 0 [ CSSD]2013-05-23 15:52:06.363 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:52:06.363 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:52:06.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:06.369 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:52:06.369 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f23c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:52:07.217 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:07.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:07.401 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:08.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:08.004 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014192, gin 1, member 0 [ CSSD]2013-05-23 15:52:08.004 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:52:08.004 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:52:08.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:08.010 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:52:08.010 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f22d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:52:09.223 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:09.319 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:09.319 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:09.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:09.474 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:11.101 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:11.102 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:52:11.103 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:11.103 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c135290) pid(21806) version 11:1:1:4 [ CSSD]2013-05-23 15:52:11.109 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:11.109 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:11.109 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:52:11.109 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:11.109 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 908 (0x7f431bfdb610) [ CSSD]2013-05-23 15:52:11.109 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb610 (RPC#908) tag(38c002c) sent to node 1 [ CSSD]2013-05-23 15:52:11.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(038c002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:11.109 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014193, gin 1, member 0 [ CSSD]2013-05-23 15:52:11.109 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:52:11.110 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:11.110 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:52:11.110 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb610 (RPC#908) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:11.110 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb610, ret 0, client 0x7f431c12e6d0 member 0x7f431c0cf650 [ CSSD]2013-05-23 15:52:11.110 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:52:11.110 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 908 [ CSSD]2013-05-23 15:52:11.116 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:11.116 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:11.116 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0cf650) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:11.116 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 909 (0x7f431bfdb690) [ CSSD]2013-05-23 15:52:11.116 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb690 (RPC#909) tag(38d002c) sent to node 1 [ CSSD]2013-05-23 15:52:11.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(038d002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:11.117 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:11.117 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb690 (RPC#909) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:11.117 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb690, ret 0, client 0x7f431c12e6d0 member 0x7f431c0cf650 [ CSSD]2013-05-23 15:52:11.117 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 909 [ CSSD]2013-05-23 15:52:11.117 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0cf650) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:11.229 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:11.319 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:11.319 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c12a550) skgpid ospid 21806 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:11.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:11.480 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:11.765 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:11.765 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:52:11.766 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:11.766 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c135290) pid(21871) version 11:1:1:4 [ CSSD]2013-05-23 15:52:11.782 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:11.782 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:11.782 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:52:11.782 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:11.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 910 (0x7f431bfdb710) [ CSSD]2013-05-23 15:52:11.782 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb710 (RPC#910) tag(38e002c) sent to node 1 [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(038e002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014194, gin 1, member 0 [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb710 (RPC#910) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb710, ret 0, client 0x7f431c12e6d0 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:52:11.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 910 [ CSSD]2013-05-23 15:52:11.789 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:11.789 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:11.789 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:11.789 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 911 (0x7f431bfdb790) [ CSSD]2013-05-23 15:52:11.789 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb790 (RPC#911) tag(38f002c) sent to node 1 [ CSSD]2013-05-23 15:52:11.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(038f002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:11.790 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:11.790 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb790 (RPC#911) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:11.790 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb790, ret 0, client 0x7f431c12e6d0 member 0x7f431c109eb0 [ CSSD]2013-05-23 15:52:11.790 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 911 [ CSSD]2013-05-23 15:52:11.790 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:11.992 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:11.992 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c11ab50) skgpid ospid 21871 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:13.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:13.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:13.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:13.724 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:13.726 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:52:13.726 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:13.726 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c135290) pid(22043) version 11:1:1:4 [ CSSD]2013-05-23 15:52:13.877 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 15:52:13.879 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:52:13.879 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 15:52:13.879 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c11fd30) pid(22057) version 11:1:1:4 [ CSSD]2013-05-23 15:52:13.895 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:52:13.897 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:52:13.900 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 15:52:13.900 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0840c0) skgpid ospid 22057 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:13.973 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(61/0x7f431c121510) [ CSSD]2013-05-23 15:52:13.973 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 61 (0x7f431c121510) [ CSSD]2013-05-23 15:52:13.973 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c121510 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:52:13.973 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:52:13.973 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 912 (0x7f431bfdb810) [ CSSD]2013-05-23 15:52:13.973 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb810 (RPC#912) tag(390002c) sent to node 1 [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0390002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014195, gin 1, member 0 [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c129890/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb810 (RPC#912) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb810, ret 0, client 0x7f431c121510 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:52:13.974 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 912 [ CSSD]2013-05-23 15:52:13.980 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 61 (0x7f431c121510) [ CSSD]2013-05-23 15:52:13.980 [1191196992] >TRACE: clssgmExitGrock: client 61 (0x7f431c121510), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:52:13.980 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:52:13.980 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 913 (0x7f431bfdb890) [ CSSD]2013-05-23 15:52:13.981 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb890 (RPC#913) tag(391002c) sent to node 1 [ CSSD]2013-05-23 15:52:13.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0391002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:13.981 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:52:13.981 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb890 (RPC#913) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:13.981 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb890, ret 0, client 0x7f431c121510 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:52:13.981 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 913 [ CSSD]2013-05-23 15:52:13.981 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:13.982 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:13.982 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c12a550) skgpid ospid 22043 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:14.322 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:14.323 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:15.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:15.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:15.496 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:16.329 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:52:16.329 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 914 (0x7f431bfdb910) [ CSSD]2013-05-23 15:52:16.330 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb910 (RPC#914) tag(392002c) sent to node 1 [ CSSD]2013-05-23 15:52:16.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:16.330 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 914 (0x7f431bfdb910) [ CSSD]2013-05-23 15:52:16.330 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb910 (RPC#914) state 6, flags 0x202 [ CSSD]2013-05-23 15:52:16.330 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb910, ret 0, client (nil) [ CSSD]2013-05-23 15:52:16.330 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 914 [ CSSD]2013-05-23 15:52:17.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:17.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:17.502 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:19.324 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:19.326 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:19.326 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:19.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:19.513 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:21.329 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:21.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:21.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:21.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:21.622 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014196, gin 1, member 0 [ CSSD]2013-05-23 15:52:21.622 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:52:21.622 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:52:21.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:21.628 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:52:21.628 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:52:23.340 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:23.572 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:23.572 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:24.329 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:24.329 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:25.345 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:25.577 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:25.577 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:26.352 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:52:26.353 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 915 (0x7f431bfdb990) [ CSSD]2013-05-23 15:52:26.353 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb990 (RPC#915) tag(393002c) sent to node 1 [ CSSD]2013-05-23 15:52:26.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:26.353 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 915 (0x7f431bfdb990) [ CSSD]2013-05-23 15:52:26.353 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb990 (RPC#915) state 6, flags 0x202 [ CSSD]2013-05-23 15:52:26.353 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb990, ret 0, client (nil) [ CSSD]2013-05-23 15:52:26.353 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 915 [ CSSD]2013-05-23 15:52:26.354 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:26.355 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:52:26.356 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:26.356 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c135290) pid(22173) version 11:1:1:4 [ CSSD]2013-05-23 15:52:26.361 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:26.361 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:26.361 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 15:52:26.361 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:26.361 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 916 (0x7f431bfdba10) [ CSSD]2013-05-23 15:52:26.361 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdba10 (RPC#916) tag(394002c) sent to node 1 [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0394002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014197, gin 1, member 0 [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdba10 (RPC#916) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdba10, ret 0, client 0x7f431c12e6d0 member 0x7f431c135050 [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:52:26.362 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 916 [ CSSD]2013-05-23 15:52:26.369 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:26.369 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:26.369 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135050) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:26.369 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 917 (0x7f431bfdba90) [ CSSD]2013-05-23 15:52:26.369 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdba90 (RPC#917) tag(395002c) sent to node 1 [ CSSD]2013-05-23 15:52:26.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0395002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:26.369 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:26.369 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdba90 (RPC#917) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:26.369 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdba90, ret 0, client 0x7f431c12e6d0 member 0x7f431c135050 [ CSSD]2013-05-23 15:52:26.370 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 917 [ CSSD]2013-05-23 15:52:26.370 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135050) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:26.572 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:26.572 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c0e5200) skgpid ospid 22173 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:27.337 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:27.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:27.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:29.332 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:29.332 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:29.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:29.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:29.695 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:31.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:31.701 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:31.701 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:33.361 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:33.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:33.712 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:34.335 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:34.335 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:35.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:35.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:35.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:36.376 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:52:36.376 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 918 (0x7f431bfdbb10) [ CSSD]2013-05-23 15:52:36.376 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbb10 (RPC#918) tag(396002c) sent to node 1 [ CSSD]2013-05-23 15:52:36.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:36.377 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 918 (0x7f431bfdbb10) [ CSSD]2013-05-23 15:52:36.377 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbb10 (RPC#918) state 6, flags 0x202 [ CSSD]2013-05-23 15:52:36.377 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbb10, ret 0, client (nil) [ CSSD]2013-05-23 15:52:36.377 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 918 [ CSSD]2013-05-23 15:52:36.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:36.880 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014198, gin 1, member 0 [ CSSD]2013-05-23 15:52:36.880 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:52:36.880 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:52:36.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:36.886 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:52:36.886 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:52:37.369 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:37.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:37.724 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:39.338 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:39.339 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:39.375 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:39.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:39.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:41.380 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:41.604 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:41.605 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:52:41.605 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:41.605 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c135290) pid(22311) version 11:1:1:4 [ CSSD]2013-05-23 15:52:41.611 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:41.611 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:41.611 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:52:41.611 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:41.611 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 919 (0x7f431bfdbb90) [ CSSD]2013-05-23 15:52:41.611 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbb90 (RPC#919) tag(397002c) sent to node 1 [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0397002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014199, gin 1, member 0 [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbb90 (RPC#919) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdbb90, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:52:41.612 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 919 [ CSSD]2013-05-23 15:52:41.618 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:41.618 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:41.618 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:41.618 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 920 (0x7f431bfdbc10) [ CSSD]2013-05-23 15:52:41.618 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbc10 (RPC#920) tag(398002c) sent to node 1 [ CSSD]2013-05-23 15:52:41.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0398002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:41.619 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:41.619 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbc10 (RPC#920) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:41.619 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbc10, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:52:41.619 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 920 [ CSSD]2013-05-23 15:52:41.619 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:41.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:41.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:41.822 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:41.822 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c0840c0) skgpid ospid 22311 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:43.341 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:43.342 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:52:43.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:43.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:43.793 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:45.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:45.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:45.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:46.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:52:46.403 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 921 (0x7f431bfdbc90) [ CSSD]2013-05-23 15:52:46.404 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbc90 (RPC#921) tag(399002c) sent to node 1 [ CSSD]2013-05-23 15:52:46.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:46.404 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 921 (0x7f431bfdbc90) [ CSSD]2013-05-23 15:52:46.404 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbc90 (RPC#921) state 6, flags 0x202 [ CSSD]2013-05-23 15:52:46.404 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbc90, ret 0, client (nil) [ CSSD]2013-05-23 15:52:46.404 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 921 [ CSSD]2013-05-23 15:52:47.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:47.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:47.804 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:48.344 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:48.345 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:49.454 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:49.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:49.816 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:51.459 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:51.822 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:51.822 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:52.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:52.132 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014200, gin 1, member 0 [ CSSD]2013-05-23 15:52:52.132 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:52:52.132 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:52:52.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:52.138 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:52:52.138 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:52:53.348 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:53.348 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:53.452 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:53.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:53.873 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:55.476 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:55.878 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:55.878 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:56.426 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:52:56.426 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 922 (0x7f431bfdbd10) [ CSSD]2013-05-23 15:52:56.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbd10 (RPC#922) tag(39a002c) sent to node 1 [ CSSD]2013-05-23 15:52:56.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:56.427 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 922 (0x7f431bfdbd10) [ CSSD]2013-05-23 15:52:56.427 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbd10 (RPC#922) state 6, flags 0x202 [ CSSD]2013-05-23 15:52:56.427 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbd10, ret 0, client (nil) [ CSSD]2013-05-23 15:52:56.427 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 922 [ CSSD]2013-05-23 15:52:56.853 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:52:56.854 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:52:56.855 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:52:56.855 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c135290) pid(22559) version 11:1:1:4 [ CSSD]2013-05-23 15:52:56.871 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:56.871 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:56.871 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:52:56.871 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:56.871 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 923 (0x7f431bfdbd90) [ CSSD]2013-05-23 15:52:56.871 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbd90 (RPC#923) tag(39b002c) sent to node 1 [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(039b002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014201, gin 1, member 0 [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbd90 (RPC#923) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdbd90, ret 0, client 0x7f431c12e6d0 member 0x7f431c108da0 [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:52:56.872 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 923 [ CSSD]2013-05-23 15:52:56.878 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:52:56.878 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:56.878 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:52:56.878 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 924 (0x7f431bfdbe10) [ CSSD]2013-05-23 15:52:56.878 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbe10 (RPC#924) tag(39c002c) sent to node 1 [ CSSD]2013-05-23 15:52:56.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(039c002c) incarnation(174631051) [ CSSD]2013-05-23 15:52:56.879 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:52:56.879 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbe10 (RPC#924) state 6, flags 0x100 [ CSSD]2013-05-23 15:52:56.879 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbe10, ret 0, client 0x7f431c12e6d0 member 0x7f431c108da0 [ CSSD]2013-05-23 15:52:56.879 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 924 [ CSSD]2013-05-23 15:52:56.879 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:52:57.081 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:52:57.081 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c11ab50) skgpid ospid 22559 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:52:57.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:57.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:57.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:52:58.351 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:52:58.351 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:52:59.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:52:59.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:52:59.916 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:01.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:01.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:01.922 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:02.353 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:02.353 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:53:03.551 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:03.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:03.933 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:05.556 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:05.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:05.939 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:06.449 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:53:06.449 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 925 (0x7f431bfdbe90) [ CSSD]2013-05-23 15:53:06.449 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbe90 (RPC#925) tag(39d002c) sent to node 1 [ CSSD]2013-05-23 15:53:06.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:06.450 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 925 (0x7f431bfdbe90) [ CSSD]2013-05-23 15:53:06.450 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbe90 (RPC#925) state 6, flags 0x202 [ CSSD]2013-05-23 15:53:06.450 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbe90, ret 0, client (nil) [ CSSD]2013-05-23 15:53:06.450 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 925 [ CSSD]2013-05-23 15:53:07.356 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:07.356 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:07.380 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:07.380 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014202, gin 1, member 0 [ CSSD]2013-05-23 15:53:07.380 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:53:07.380 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:53:07.387 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:07.387 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:53:07.387 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f23c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:53:07.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:07.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:07.945 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:08.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:08.270 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014203, gin 1, member 0 [ CSSD]2013-05-23 15:53:08.270 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:53:08.270 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:53:08.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:08.276 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:53:08.276 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:53:09.576 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:09.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:09.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:11.581 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:12.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:12.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:12.032 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:53:12.033 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:53:12.034 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:53:12.034 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c135290) pid(22698) version 11:1:1:4 [ CSSD]2013-05-23 15:53:12.040 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:53:12.040 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:53:12.040 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:53:12.040 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:12.040 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 926 (0x7f431bfdbf10) [ CSSD]2013-05-23 15:53:12.040 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbf10 (RPC#926) tag(39e002c) sent to node 1 [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(039e002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014204, gin 1, member 0 [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbf10 (RPC#926) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdbf10, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:53:12.041 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 926 [ CSSD]2013-05-23 15:53:12.048 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:53:12.048 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:12.048 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:12.048 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 927 (0x7f431bfdbf90) [ CSSD]2013-05-23 15:53:12.048 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbf90 (RPC#927) tag(39f002c) sent to node 1 [ CSSD]2013-05-23 15:53:12.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(039f002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:12.048 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:12.048 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbf90 (RPC#927) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:12.048 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbf90, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e6d60 [ CSSD]2013-05-23 15:53:12.048 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 927 [ CSSD]2013-05-23 15:53:12.048 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:53:12.117 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 15:53:12.118 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:53:12.118 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 15:53:12.118 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c11fd30) pid(22707) version 11:1:1:4 [ CSSD]2013-05-23 15:53:12.134 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 15:53:12.134 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 15:53:12.134 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:53:12.134 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:12.134 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 928 (0x7f431bfdc010) [ CSSD]2013-05-23 15:53:12.134 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc010 (RPC#928) tag(3a0002c) sent to node 1 [ CSSD]2013-05-23 15:53:12.134 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03a0002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:12.134 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014205, gin 1, member 0 [ CSSD]2013-05-23 15:53:12.134 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:53:12.135 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:53:12.135 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:53:12.135 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc010 (RPC#928) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:12.135 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc010, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 15:53:12.135 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 15:53:12.135 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 928 [ CSSD]2013-05-23 15:53:12.140 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 15:53:12.140 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:12.140 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:12.140 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 929 (0x7f431bfdc090) [ CSSD]2013-05-23 15:53:12.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc090 (RPC#929) tag(3a1002c) sent to node 1 [ CSSD]2013-05-23 15:53:12.141 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03a1002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:12.141 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:12.141 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc090 (RPC#929) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:12.141 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc090, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 15:53:12.141 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 929 [ CSSD]2013-05-23 15:53:12.141 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:53:12.251 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:53:12.251 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c0e5200) skgpid ospid 22698 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:12.344 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 15:53:12.344 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0840c0) skgpid ospid 22707 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:12.359 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:12.359 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:13.633 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:14.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:14.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:15.638 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:16.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:16.019 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:16.479 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:53:16.479 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 930 (0x7f431bfdc110) [ CSSD]2013-05-23 15:53:16.479 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc110 (RPC#930) tag(3a2002c) sent to node 1 [ CSSD]2013-05-23 15:53:16.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:16.480 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 930 (0x7f431bfdc110) [ CSSD]2013-05-23 15:53:16.480 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc110 (RPC#930) state 6, flags 0x202 [ CSSD]2013-05-23 15:53:16.480 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc110, ret 0, client (nil) [ CSSD]2013-05-23 15:53:16.480 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 930 [ CSSD]2013-05-23 15:53:17.362 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:17.362 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:17.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:18.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:18.025 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:19.656 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:20.037 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:20.037 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:21.365 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:21.365 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:53:21.661 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:22.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:22.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:22.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:22.629 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014206, gin 1, member 0 [ CSSD]2013-05-23 15:53:22.629 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1fd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:53:22.629 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:53:22.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:22.635 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:53:22.635 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1d40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:53:23.673 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:24.115 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:24.115 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:25.679 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:26.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:26.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:26.368 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:26.368 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:26.503 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:53:26.504 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 931 (0x7f431bfdc190) [ CSSD]2013-05-23 15:53:26.504 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc190 (RPC#931) tag(3a3002c) sent to node 1 [ CSSD]2013-05-23 15:53:26.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:26.504 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 931 (0x7f431bfdc190) [ CSSD]2013-05-23 15:53:26.504 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc190 (RPC#931) state 6, flags 0x202 [ CSSD]2013-05-23 15:53:26.504 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc190, ret 0, client (nil) [ CSSD]2013-05-23 15:53:26.505 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 931 [ CSSD]2013-05-23 15:53:27.376 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:53:27.376 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:53:27.377 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:53:27.377 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c135290) pid(22888) version 11:1:1:4 [ CSSD]2013-05-23 15:53:27.382 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:53:27.382 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:53:27.382 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 15:53:27.383 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:27.383 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 932 (0x7f431bfdc210) [ CSSD]2013-05-23 15:53:27.383 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc210 (RPC#932) tag(3a4002c) sent to node 1 [ CSSD]2013-05-23 15:53:27.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03a4002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:27.383 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014207, gin 1, member 0 [ CSSD]2013-05-23 15:53:27.383 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129890/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:53:27.383 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:53:27.384 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:53:27.384 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc210 (RPC#932) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:27.384 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc210, ret 0, client 0x7f431c12e6d0 member 0x7f431c135050 [ CSSD]2013-05-23 15:53:27.384 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:53:27.384 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 932 [ CSSD]2013-05-23 15:53:27.390 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:53:27.390 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:27.390 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135050) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:27.390 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 933 (0x7f431bfdc290) [ CSSD]2013-05-23 15:53:27.390 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc290 (RPC#933) tag(3a5002c) sent to node 1 [ CSSD]2013-05-23 15:53:27.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03a5002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:27.391 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:27.391 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc290 (RPC#933) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:27.391 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc290, ret 0, client 0x7f431c12e6d0 member 0x7f431c135050 [ CSSD]2013-05-23 15:53:27.391 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 933 [ CSSD]2013-05-23 15:53:27.391 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135050) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:53:27.593 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:53:27.593 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c0e6720) skgpid ospid 22888 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:27.731 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:28.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:28.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:29.755 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:30.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:30.137 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:31.371 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:31.372 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:31.761 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:32.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:32.143 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:33.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:34.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:34.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:34.997 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 15:53:34.998 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:53:34.999 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 15:53:34.999 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c135290) pid(21317/21317) version 11:1:1:4 [ CSSD]2013-05-23 15:53:35.014 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(1/0x22f2350) [ CSSD]2013-05-23 15:53:35.014 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f2350) [ CSSD]2013-05-23 15:53:35.014 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:53:35.014 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 15:53:35.779 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:36.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:36.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:36.375 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:36.375 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:36.530 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:53:36.530 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 934 (0x7f431bfdc310) [ CSSD]2013-05-23 15:53:36.531 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc310 (RPC#934) tag(3a6002c) sent to node 1 [ CSSD]2013-05-23 15:53:36.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:36.531 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 934 (0x7f431bfdc310) [ CSSD]2013-05-23 15:53:36.531 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc310 (RPC#934) state 6, flags 0x202 [ CSSD]2013-05-23 15:53:36.531 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc310, ret 0, client (nil) [ CSSD]2013-05-23 15:53:36.531 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 934 [ CSSD]2013-05-23 15:53:36.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:36.568 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014208, gin 1, member 0 [ CSSD]2013-05-23 15:53:36.568 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f24c0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 15:53:36.568 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:53:36.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:36.575 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 15:53:36.575 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22f27c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:53:36.700 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:53:36.700 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:53:36.702 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:53:36.703 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:53:36.705 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:53:36.705 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:53:37.831 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:37.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:37.879 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014209, gin 1, member 0 [ CSSD]2013-05-23 15:53:37.879 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f24c0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:53:37.879 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:53:37.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:37.885 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:53:37.885 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f27c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:53:38.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:38.207 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:39.856 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:40.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:40.236 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:40.738 [1191196992] >TRACE: clssgmAllocProc: (0x23178a0) allocated [ CSSD]2013-05-23 15:53:40.738 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:53:40.739 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23178a0 - 1,2,3 [ CSSD]2013-05-23 15:53:40.739 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2150) proc(0x23178a0) pid(21313/21313) version 11:1:1:4 [ CSSD]2013-05-23 15:53:40.744 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x23178a0), client(1/0x22ee1b0) [ CSSD]2013-05-23 15:53:40.744 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:53:40.744 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:53:40.744 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 15:53:40.982 [1191196992] >TRACE: clssgmAllocProc: (0x23191f0) allocated [ CSSD]2013-05-23 15:53:40.983 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:53:40.984 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191f0 - 1,2,3 [ CSSD]2013-05-23 15:53:40.984 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2318360) proc(0x23191f0) pid(22468/22468) version 11:1:1:4 [ CSSD]2013-05-23 15:53:40.999 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x23191f0), client(1/0x2319520) [ CSSD]2013-05-23 15:53:40.999 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2319520) [ CSSD]2013-05-23 15:53:40.999 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:53:40.999 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 15:53:41.378 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:41.378 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:41.721 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x23191f0), client(2/0x22e4ef0) [ CSSD]2013-05-23 15:53:41.721 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22e4ef0) [ CSSD]2013-05-23 15:53:41.721 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:53:41.721 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 15:53:41.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:42.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:42.261 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:42.635 [1191196992] >TRACE: clssgmAllocProc: (0x22e75d0) allocated [ CSSD]2013-05-23 15:53:42.636 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:53:42.637 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e75d0 - 1,2,3 [ CSSD]2013-05-23 15:53:42.637 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2320f80) proc(0x22e75d0) pid(23049) version 11:1:1:4 [ CSSD]2013-05-23 15:53:42.653 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e75d0), client(1/0x22e7d00) [ CSSD]2013-05-23 15:53:42.653 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e7d00) [ CSSD]2013-05-23 15:53:42.653 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e7d00 with con 0x2323170, requested num -1 [ CSSD]2013-05-23 15:53:42.653 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:42.653 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 935 (0x7f431bfdc390) [ CSSD]2013-05-23 15:53:42.653 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc390 (RPC#935) tag(3a7002c) sent to node 1 [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03a7002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014210, gin 1, member 0 [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e7e70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc390 (RPC#935) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc390, ret 0, client 0x22e7d00 member 0x22e8170 [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e7d00 [ CSSD]2013-05-23 15:53:42.654 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 935 [ CSSD]2013-05-23 15:53:42.661 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e7d00) [ CSSD]2013-05-23 15:53:42.661 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e7d00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:42.661 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8170) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:42.661 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 936 (0x7f431bfdc410) [ CSSD]2013-05-23 15:53:42.661 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc410 (RPC#936) tag(3a8002c) sent to node 1 [ CSSD]2013-05-23 15:53:42.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03a8002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:42.661 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:42.661 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc410 (RPC#936) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:42.662 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc410, ret 0, client 0x22e7d00 member 0x22e8170 [ CSSD]2013-05-23 15:53:42.662 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 936 [ CSSD]2013-05-23 15:53:42.662 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8170) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:53:42.864 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e75d0 [ CSSD]2013-05-23 15:53:42.864 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e75d0) con(0x2320f80) skgpid ospid 23049 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:43.873 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:44.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:44.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:45.879 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:46.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:46.301 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:46.381 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:46.381 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:46.573 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:53:46.573 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 937 (0x7f431bfdc490) [ CSSD]2013-05-23 15:53:46.573 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc490 (RPC#937) tag(3a9002c) sent to node 1 [ CSSD]2013-05-23 15:53:46.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:46.574 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 937 (0x7f431bfdc490) [ CSSD]2013-05-23 15:53:46.574 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc490 (RPC#937) state 6, flags 0x202 [ CSSD]2013-05-23 15:53:46.574 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc490, ret 0, client (nil) [ CSSD]2013-05-23 15:53:46.574 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 937 [ CSSD]2013-05-23 15:53:47.334 [1191196992] >TRACE: clssgmAllocProc: (0x22e75d0) allocated [ CSSD]2013-05-23 15:53:47.335 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:53:47.335 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e75d0 - 1,2,3 [ CSSD]2013-05-23 15:53:47.335 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2320f80) proc(0x22e75d0) pid(22466/22466) version 11:1:1:4 [ CSSD]2013-05-23 15:53:47.351 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e75d0), client(1/0x22e7b70) [ CSSD]2013-05-23 15:53:47.351 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e7b70) [ CSSD]2013-05-23 15:53:47.351 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:53:47.351 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 15:53:47.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:47.972 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135290), client(2/0x22e8180) [ CSSD]2013-05-23 15:53:47.972 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22e8180) [ CSSD]2013-05-23 15:53:47.972 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:53:47.972 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 30 [ CSSD]2013-05-23 15:53:48.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:48.327 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:49.955 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:50.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:50.338 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:51.384 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:51.384 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:51.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:52.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:52.343 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:53.067 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f4200) [ CSSD]2013-05-23 15:53:53.067 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f4200), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:53:53.067 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f4200), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:53:53.067 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 29 [ CSSD]2013-05-23 15:53:53.071 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22e2eb0) [ CSSD]2013-05-23 15:53:53.071 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22e2eb0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:53:53.071 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22e2eb0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:53:53.071 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 15:53:53.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ef070) [ CSSD]2013-05-23 15:53:53.072 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ef070), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:53:53.072 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ef070), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:53:53.072 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 15:53:53.072 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:53:53.072 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x205ed50) skgpid 17603 ospid 17603 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:53.078 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x205ecb0) [ CSSD]2013-05-23 15:53:53.078 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x205ecb0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:53:53.078 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x205ecb0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:53:53.078 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 15:53:53.078 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c121750 [ CSSD]2013-05-23 15:53:53.078 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c121750) con(0x7f431c0e5900) skgpid 17605 ospid 17605 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:53.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:53.130 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014211, gin 1, member 0 [ CSSD]2013-05-23 15:53:53.130 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:53:53.130 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:53:53.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:53.138 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:53:53.138 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f280) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:53:53.975 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:54.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:54.396 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:55.980 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:56.387 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:53:56.387 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:53:56.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:56.402 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:53:56.599 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:53:56.599 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 938 (0x7f431bfdc510) [ CSSD]2013-05-23 15:53:56.600 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc510 (RPC#938) tag(3aa002c) sent to node 1 [ CSSD]2013-05-23 15:53:56.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:56.600 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 938 (0x7f431bfdc510) [ CSSD]2013-05-23 15:53:56.600 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc510 (RPC#938) state 6, flags 0x202 [ CSSD]2013-05-23 15:53:56.600 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc510, ret 0, client (nil) [ CSSD]2013-05-23 15:53:56.600 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 938 [ CSSD]2013-05-23 15:53:57.895 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:53:57.896 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:53:57.897 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:53:57.897 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2eb0) proc(0x22e01e0) pid(23208) version 11:1:1:4 [ CSSD]2013-05-23 15:53:57.903 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:53:57.903 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:53:57.903 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x22e7ce0, requested num -1 [ CSSD]2013-05-23 15:53:57.903 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:57.903 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 939 (0x7f431bfdc590) [ CSSD]2013-05-23 15:53:57.903 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc590 (RPC#939) tag(3ab002c) sent to node 1 [ CSSD]2013-05-23 15:53:57.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ab002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:57.903 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014212, gin 1, member 0 [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc590 (RPC#939) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc590, ret 0, client 0x22f4200 member 0x22fdb60 [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:53:57.904 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 939 [ CSSD]2013-05-23 15:53:57.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:53:57.910 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:57.910 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:53:57.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 940 (0x7f431bfdc610) [ CSSD]2013-05-23 15:53:57.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc610 (RPC#940) tag(3ac002c) sent to node 1 [ CSSD]2013-05-23 15:53:57.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ac002c) incarnation(174631051) [ CSSD]2013-05-23 15:53:57.911 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:53:57.911 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc610 (RPC#940) state 6, flags 0x100 [ CSSD]2013-05-23 15:53:57.911 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc610, ret 0, client 0x22f4200 member 0x22fdb60 [ CSSD]2013-05-23 15:53:57.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 940 [ CSSD]2013-05-23 15:53:57.911 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:53:58.002 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:53:58.113 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:53:58.113 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22e2eb0) skgpid ospid 23208 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:53:58.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:53:58.448 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:00.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:00.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:00.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:01.390 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:01.390 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:02.012 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:02.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:02.545 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:04.045 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:04.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:04.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:05.393 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:05.393 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:54:06.051 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:06.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:06.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:06.623 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:54:06.623 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 941 (0x7f431bfdc690) [ CSSD]2013-05-23 15:54:06.623 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc690 (RPC#941) tag(3ad002c) sent to node 1 [ CSSD]2013-05-23 15:54:06.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:06.623 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 941 (0x7f431bfdc690) [ CSSD]2013-05-23 15:54:06.623 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc690 (RPC#941) state 6, flags 0x202 [ CSSD]2013-05-23 15:54:06.623 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc690, ret 0, client (nil) [ CSSD]2013-05-23 15:54:06.624 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 941 [ CSSD]2013-05-23 15:54:08.062 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:08.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:08.382 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014213, gin 1, member 0 [ CSSD]2013-05-23 15:54:08.382 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:54:08.382 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:54:08.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:08.389 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:54:08.389 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f280) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:54:08.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:08.532 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014214, gin 1, member 0 [ CSSD]2013-05-23 15:54:08.532 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:54:08.532 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:54:08.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:08.539 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:54:08.539 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:54:08.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:08.614 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:10.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:10.396 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:10.396 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:10.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:10.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:12.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:12.301 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:54:12.302 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:54:12.302 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:54:12.302 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302f30) proc(0x22e01e0) pid(23345) version 11:1:1:4 [ CSSD]2013-05-23 15:54:12.308 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:54:12.308 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:12.308 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x205ecb0, requested num -1 [ CSSD]2013-05-23 15:54:12.308 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:12.308 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 942 (0x7f431bfdc710) [ CSSD]2013-05-23 15:54:12.308 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc710 (RPC#942) tag(3ae002c) sent to node 1 [ CSSD]2013-05-23 15:54:12.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ae002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:12.308 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014215, gin 1, member 0 [ CSSD]2013-05-23 15:54:12.308 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:54:12.309 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:54:12.309 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:54:12.309 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc710 (RPC#942) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:12.309 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc710, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:54:12.309 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:54:12.309 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 942 [ CSSD]2013-05-23 15:54:12.315 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:12.315 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:12.315 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2317510) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:12.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 943 (0x7f431bfdc790) [ CSSD]2013-05-23 15:54:12.315 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc790 (RPC#943) tag(3af002c) sent to node 1 [ CSSD]2013-05-23 15:54:12.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03af002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:12.315 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:12.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc790 (RPC#943) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:12.315 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc790, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:54:12.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 943 [ CSSD]2013-05-23 15:54:12.316 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2317510) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:54:12.518 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:54:12.518 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x2302f30) skgpid ospid 23345 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:54:12.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:12.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:13.149 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:54:13.151 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:54:13.151 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:54:13.151 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2eb0) proc(0x22e01e0) pid(23360) version 11:1:1:4 [ CSSD]2013-05-23 15:54:13.157 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:54:13.157 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:13.157 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x22e7ce0, requested num -1 [ CSSD]2013-05-23 15:54:13.157 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:13.157 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 944 (0x7f431bfdc810) [ CSSD]2013-05-23 15:54:13.158 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc810 (RPC#944) tag(3b0002c) sent to node 1 [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03b0002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014216, gin 1, member 0 [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc810 (RPC#944) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc810, ret 0, client 0x22f4200 member 0x22fc9c0 [ CSSD]2013-05-23 15:54:13.158 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:54:13.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 944 [ CSSD]2013-05-23 15:54:13.165 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:13.165 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:13.165 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc9c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:13.165 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 945 (0x7f431bfdc890) [ CSSD]2013-05-23 15:54:13.165 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc890 (RPC#945) tag(3b1002c) sent to node 1 [ CSSD]2013-05-23 15:54:13.165 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03b1002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:13.165 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:13.165 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc890 (RPC#945) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:13.165 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc890, ret 0, client 0x22f4200 member 0x22fc9c0 [ CSSD]2013-05-23 15:54:13.166 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 945 [ CSSD]2013-05-23 15:54:13.166 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc9c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:54:13.368 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:54:13.368 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22e2eb0) skgpid ospid 23360 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:54:14.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:14.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:14.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:15.399 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:15.399 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:16.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:16.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:16.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:16.665 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:54:16.666 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 946 (0x7f431bfdc910) [ CSSD]2013-05-23 15:54:16.666 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc910 (RPC#946) tag(3b2002c) sent to node 1 [ CSSD]2013-05-23 15:54:16.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:16.666 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 946 (0x7f431bfdc910) [ CSSD]2013-05-23 15:54:16.666 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc910 (RPC#946) state 6, flags 0x202 [ CSSD]2013-05-23 15:54:16.666 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc910, ret 0, client (nil) [ CSSD]2013-05-23 15:54:16.666 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 946 [ CSSD]2013-05-23 15:54:18.181 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:18.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:18.645 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:20.186 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:20.401 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:20.401 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:20.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:20.651 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:22.241 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:22.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:22.657 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:23.523 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:54:23.523 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:54:23.524 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:54:23.524 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:54:23.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:23.606 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014217, gin 1, member 0 [ CSSD]2013-05-23 15:54:23.606 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x2303160/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 15:54:23.606 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 15:54:23.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:23.612 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 15:54:23.612 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x230f280) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:54:23.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:23.651 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014218, gin 1, member 0 [ CSSD]2013-05-23 15:54:23.651 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:54:23.651 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:54:23.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:23.658 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:54:23.658 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:54:24.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:24.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:24.707 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:25.404 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:25.404 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:26.272 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:26.690 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:54:26.691 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 947 (0x7f431bfdc990) [ CSSD]2013-05-23 15:54:26.691 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc990 (RPC#947) tag(3b3002c) sent to node 1 [ CSSD]2013-05-23 15:54:26.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:26.691 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 947 (0x7f431bfdc990) [ CSSD]2013-05-23 15:54:26.691 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc990 (RPC#947) state 6, flags 0x202 [ CSSD]2013-05-23 15:54:26.691 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc990, ret 0, client (nil) [ CSSD]2013-05-23 15:54:26.692 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 947 [ CSSD]2013-05-23 15:54:26.713 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:26.713 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:28.283 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:28.400 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:54:28.401 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:54:28.402 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:54:28.402 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2fc0) proc(0x22e01e0) pid(23521) version 11:1:1:4 [ CSSD]2013-05-23 15:54:28.418 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:54:28.418 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:28.418 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x22eefa0, requested num -1 [ CSSD]2013-05-23 15:54:28.418 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:28.418 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 948 (0x7f431bfdca10) [ CSSD]2013-05-23 15:54:28.418 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdca10 (RPC#948) tag(3b4002c) sent to node 1 [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03b4002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014219, gin 1, member 0 [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdca10 (RPC#948) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdca10, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:54:28.419 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:54:28.420 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 948 [ CSSD]2013-05-23 15:54:28.427 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:28.427 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:28.427 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2317510) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:28.427 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 949 (0x7f431bfdca90) [ CSSD]2013-05-23 15:54:28.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdca90 (RPC#949) tag(3b5002c) sent to node 1 [ CSSD]2013-05-23 15:54:28.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03b5002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:28.428 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:28.428 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdca90 (RPC#949) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:28.428 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdca90, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:54:28.428 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 949 [ CSSD]2013-05-23 15:54:28.428 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2317510) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:54:28.630 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:54:28.630 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22e2fc0) skgpid ospid 23521 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:54:28.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:28.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:29.406 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:29.407 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:54:30.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:30.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:30.731 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:32.341 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:32.737 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:32.737 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:34.347 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:34.410 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:34.410 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:34.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:34.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:36.370 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:36.719 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:54:36.719 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 950 (0x7f431bfdcb10) [ CSSD]2013-05-23 15:54:36.719 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcb10 (RPC#950) tag(3b6002c) sent to node 1 [ CSSD]2013-05-23 15:54:36.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:36.719 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 950 (0x7f431bfdcb10) [ CSSD]2013-05-23 15:54:36.719 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcb10 (RPC#950) state 6, flags 0x202 [ CSSD]2013-05-23 15:54:36.719 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcb10, ret 0, client (nil) [ CSSD]2013-05-23 15:54:36.720 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 950 [ CSSD]2013-05-23 15:54:36.754 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:36.754 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:38.384 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:38.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:38.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:38.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:38.902 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014220, gin 1, member 0 [ CSSD]2013-05-23 15:54:38.902 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:54:38.902 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:54:38.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:38.909 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:54:38.909 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:54:39.413 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:39.413 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:40.409 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:40.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:40.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:42.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:42.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:42.837 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:43.662 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:54:43.664 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:54:43.664 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:54:43.664 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2eb0) proc(0x22e01e0) pid(23665) version 11:1:1:4 [ CSSD]2013-05-23 15:54:43.670 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:54:43.670 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:43.670 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x22e7ce0, requested num -1 [ CSSD]2013-05-23 15:54:43.670 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:43.670 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 951 (0x7f431bfdcb90) [ CSSD]2013-05-23 15:54:43.671 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcb90 (RPC#951) tag(3b7002c) sent to node 1 [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03b7002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014221, gin 1, member 0 [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcb90 (RPC#951) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdcb90, ret 0, client 0x22f4200 member 0x22fc9c0 [ CSSD]2013-05-23 15:54:43.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:54:43.672 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 951 [ CSSD]2013-05-23 15:54:43.678 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:43.678 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:43.678 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc9c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:43.678 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 952 (0x7f431bfdcc10) [ CSSD]2013-05-23 15:54:43.678 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcc10 (RPC#952) tag(3b8002c) sent to node 1 [ CSSD]2013-05-23 15:54:43.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03b8002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:43.678 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:43.678 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcc10 (RPC#952) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:43.678 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdcc10, ret 0, client 0x22f4200 member 0x22fc9c0 [ CSSD]2013-05-23 15:54:43.679 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 952 [ CSSD]2013-05-23 15:54:43.679 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc9c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:54:43.881 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:54:43.881 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22e2eb0) skgpid ospid 23665 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:54:44.416 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:44.416 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:44.447 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:44.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:44.847 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:46.474 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:46.745 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:54:46.745 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 953 (0x7f431bfdcc90) [ CSSD]2013-05-23 15:54:46.745 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcc90 (RPC#953) tag(3b9002c) sent to node 1 [ CSSD]2013-05-23 15:54:46.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:46.746 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 953 (0x7f431bfdcc90) [ CSSD]2013-05-23 15:54:46.746 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcc90 (RPC#953) state 6, flags 0x202 [ CSSD]2013-05-23 15:54:46.746 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcc90, ret 0, client (nil) [ CSSD]2013-05-23 15:54:46.746 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 953 [ CSSD]2013-05-23 15:54:46.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:46.853 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:48.419 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:48.419 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:54:48.484 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:48.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:48.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:50.489 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:50.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:50.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:52.542 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:52.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:52.937 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:53.421 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:53.422 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:54.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:54.151 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014222, gin 1, member 0 [ CSSD]2013-05-23 15:54:54.152 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:54:54.152 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:54:54.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:54.158 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:54:54.158 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c135050) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:54:54.567 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:54.947 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:54.947 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:56.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:56.775 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:54:56.775 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 954 (0x7f431bfdcd10) [ CSSD]2013-05-23 15:54:56.776 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcd10 (RPC#954) tag(3ba002c) sent to node 1 [ CSSD]2013-05-23 15:54:56.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:56.776 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 954 (0x7f431bfdcd10) [ CSSD]2013-05-23 15:54:56.776 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcd10 (RPC#954) state 6, flags 0x202 [ CSSD]2013-05-23 15:54:56.776 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcd10, ret 0, client (nil) [ CSSD]2013-05-23 15:54:56.776 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 954 [ CSSD]2013-05-23 15:54:56.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:56.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:58.425 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:54:58.425 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:54:58.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:54:58.913 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:54:58.914 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:54:58.914 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:54:58.914 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302f30) proc(0x22e01e0) pid(23827) version 11:1:1:4 [ CSSD]2013-05-23 15:54:58.920 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:54:58.920 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:58.920 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x205ecb0, requested num -1 [ CSSD]2013-05-23 15:54:58.920 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:58.920 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 955 (0x7f431bfdcd90) [ CSSD]2013-05-23 15:54:58.920 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcd90 (RPC#955) tag(3bb002c) sent to node 1 [ CSSD]2013-05-23 15:54:58.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03bb002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:58.920 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014223, gin 1, member 0 [ CSSD]2013-05-23 15:54:58.920 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:54:58.921 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:54:58.921 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:54:58.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcd90 (RPC#955) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:58.921 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdcd90, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:54:58.921 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:54:58.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 955 [ CSSD]2013-05-23 15:54:58.927 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:54:58.927 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:58.927 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2317510) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:54:58.927 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 956 (0x7f431bfdce10) [ CSSD]2013-05-23 15:54:58.927 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdce10 (RPC#956) tag(3bc002c) sent to node 1 [ CSSD]2013-05-23 15:54:58.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03bc002c) incarnation(174631051) [ CSSD]2013-05-23 15:54:58.928 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:54:58.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdce10 (RPC#956) state 6, flags 0x100 [ CSSD]2013-05-23 15:54:58.928 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdce10, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:54:58.928 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 956 [ CSSD]2013-05-23 15:54:58.928 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2317510) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:54:59.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:54:59.006 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:54:59.130 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:54:59.130 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x2302f30) skgpid ospid 23827 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:00.610 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:01.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:01.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:02.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:03.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:03.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:03.428 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:03.428 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:04.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:05.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:05.029 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:06.695 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:06.802 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:55:06.802 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 957 (0x7f431bfdce90) [ CSSD]2013-05-23 15:55:06.802 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdce90 (RPC#957) tag(3bd002c) sent to node 1 [ CSSD]2013-05-23 15:55:06.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:06.802 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 957 (0x7f431bfdce90) [ CSSD]2013-05-23 15:55:06.802 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdce90 (RPC#957) state 6, flags 0x202 [ CSSD]2013-05-23 15:55:06.802 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdce90, ret 0, client (nil) [ CSSD]2013-05-23 15:55:06.803 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 957 [ CSSD]2013-05-23 15:55:07.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:07.055 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:07.431 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:07.431 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:55:07.519 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:55:07.520 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:55:07.521 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:55:07.521 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3fe0) proc(0x22e01e0) pid(23936) version 11:1:1:4 [ CSSD]2013-05-23 15:55:08.223 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:55:08.223 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22f3fe0) skgpid ospid 23936 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:08.766 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:08.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:08.790 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014224, gin 1, member 0 [ CSSD]2013-05-23 15:55:08.790 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:55:08.790 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:55:08.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:08.797 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:55:08.797 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c135050) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:55:09.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:09.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:09.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:09.409 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014225, gin 1, member 0 [ CSSD]2013-05-23 15:55:09.409 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:55:09.409 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:55:09.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:09.416 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:55:09.416 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:55:10.772 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:11.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:11.112 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:12.434 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:12.434 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:12.553 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:55:12.554 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:55:12.554 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:55:12.554 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2eb0) proc(0x22e01e0) pid(24066) version 11:1:1:4 [ CSSD]2013-05-23 15:55:12.560 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:55:12.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:55:12.560 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x22e7ce0, requested num -1 [ CSSD]2013-05-23 15:55:12.560 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:12.560 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 958 (0x7f431bfdcf10) [ CSSD]2013-05-23 15:55:12.560 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcf10 (RPC#958) tag(3be002c) sent to node 1 [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03be002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014226, gin 1, member 0 [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcf10 (RPC#958) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdcf10, ret 0, client 0x22f4200 member 0x22fc9c0 [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:55:12.561 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 958 [ CSSD]2013-05-23 15:55:12.567 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:55:12.567 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:12.567 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc9c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:12.567 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 959 (0x7f431bfdcf90) [ CSSD]2013-05-23 15:55:12.567 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcf90 (RPC#959) tag(3bf002c) sent to node 1 [ CSSD]2013-05-23 15:55:12.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03bf002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:12.568 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:12.568 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcf90 (RPC#959) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:12.568 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdcf90, ret 0, client 0x22f4200 member 0x22fc9c0 [ CSSD]2013-05-23 15:55:12.568 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 959 [ CSSD]2013-05-23 15:55:12.568 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc9c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:55:12.770 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:55:12.770 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22e2eb0) skgpid ospid 24066 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:12.781 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:13.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:13.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:14.162 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 15:55:14.164 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:55:14.164 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 15:55:14.164 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2eb0) proc(0x22e01e0) pid(24088) version 11:1:1:4 [ CSSD]2013-05-23 15:55:14.180 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22f4200) [ CSSD]2013-05-23 15:55:14.180 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:55:14.180 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4200 with con 0x22e7ce0, requested num -1 [ CSSD]2013-05-23 15:55:14.180 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:14.180 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 960 (0x7f431bfdd010) [ CSSD]2013-05-23 15:55:14.180 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd010 (RPC#960) tag(3c0002c) sent to node 1 [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03c0002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014227, gin 1, member 0 [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e82f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd010 (RPC#960) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd010, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4200 [ CSSD]2013-05-23 15:55:14.181 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 960 [ CSSD]2013-05-23 15:55:14.188 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4200) [ CSSD]2013-05-23 15:55:14.188 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4200), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:14.188 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2317510) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:14.188 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 961 (0x7f431bfdd090) [ CSSD]2013-05-23 15:55:14.188 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd090 (RPC#961) tag(3c1002c) sent to node 1 [ CSSD]2013-05-23 15:55:14.188 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03c1002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:14.188 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:14.188 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd090 (RPC#961) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:14.188 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd090, ret 0, client 0x22f4200 member 0x2317510 [ CSSD]2013-05-23 15:55:14.189 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 961 [ CSSD]2013-05-23 15:55:14.189 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2317510) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:55:14.391 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 15:55:14.391 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22e2eb0) skgpid ospid 24088 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:14.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:15.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:15.130 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:16.792 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:16.826 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:55:16.826 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 962 (0x7f431bfdd110) [ CSSD]2013-05-23 15:55:16.827 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd110 (RPC#962) tag(3c2002c) sent to node 1 [ CSSD]2013-05-23 15:55:16.827 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:16.827 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 962 (0x7f431bfdd110) [ CSSD]2013-05-23 15:55:16.827 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd110 (RPC#962) state 6, flags 0x202 [ CSSD]2013-05-23 15:55:16.827 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd110, ret 0, client (nil) [ CSSD]2013-05-23 15:55:16.827 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 962 [ CSSD]2013-05-23 15:55:17.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:17.136 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:17.437 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:17.437 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:18.798 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f4160) [ CSSD]2013-05-23 15:55:18.798 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f4160), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:55:18.798 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f4160), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:55:18.798 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 27 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x23024d0) [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x23024d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x23024d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2302dc0) [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2302dc0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2302dc0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7620 [ CSSD]2013-05-23 15:55:18.803 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7620) con(0x205e170) skgpid 18739 ospid 18739 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f7f80) [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f7f80), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f7f80), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ff8b0) [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ff8b0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ff8b0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302b40 [ CSSD]2013-05-23 15:55:18.806 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302b40) con(0x22ff670) skgpid 18741 ospid 18741 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ff530) [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ff530), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ff530), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090cf0 [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090cf0) con(0x7f431c12f0c0) skgpid 18743 ospid 18743 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:18.816 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:19.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:19.147 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:20.822 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:21.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:21.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:22.440 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:22.440 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:22.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:23.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:23.178 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:24.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:24.658 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014228, gin 1, member 0 [ CSSD]2013-05-23 15:55:24.658 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:55:24.658 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:55:24.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:24.665 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:55:24.665 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c109eb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:55:24.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:24.938 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 15:55:24.939 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:55:24.939 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 15:55:24.939 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22fc9c0) pid(23175/23175) version 11:1:1:4 [ CSSD]2013-05-23 15:55:24.955 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22fc9c0), client(1/0x2312950) [ CSSD]2013-05-23 15:55:24.955 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 15:55:24.955 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:55:24.955 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 15:55:24.974 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 15:55:24.974 [1191196992] >TRACE: clssgmAllocProc: (0x22ff670) allocated [ CSSD]2013-05-23 15:55:24.975 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:55:24.975 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:55:24.975 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 15:55:24.975 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7420) proc(0x22d7750) pid(24177/24177) version 11:1:1:4 [ CSSD]2013-05-23 15:55:24.976 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ff670 - 1,2,3 [ CSSD]2013-05-23 15:55:24.976 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2303710) proc(0x22ff670) pid(24175/24175) version 11:1:1:4 [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d7750), client(1/0x205ed60) [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x205ed60) [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ff670), client(1/0x2302d90) [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2302d90) [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:55:24.992 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 15:55:25.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:25.210 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:26.850 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:55:26.850 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 963 (0x7f431bfdd190) [ CSSD]2013-05-23 15:55:26.850 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd190 (RPC#963) tag(3c3002c) sent to node 1 [ CSSD]2013-05-23 15:55:26.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:26.851 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 963 (0x7f431bfdd190) [ CSSD]2013-05-23 15:55:26.851 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd190 (RPC#963) state 6, flags 0x202 [ CSSD]2013-05-23 15:55:26.851 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd190, ret 0, client (nil) [ CSSD]2013-05-23 15:55:26.851 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 963 [ CSSD]2013-05-23 15:55:26.885 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:27.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:27.235 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:27.444 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:27.444 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:28.487 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d7750), client(2/0x23026c0) [ CSSD]2013-05-23 15:55:28.488 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x23026c0) [ CSSD]2013-05-23 15:55:28.488 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:55:28.488 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 15:55:28.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:29.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:29.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:29.424 [1191196992] >TRACE: clssgmAllocProc: (0x22f4fa0) allocated [ CSSD]2013-05-23 15:55:29.426 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:55:29.426 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4fa0 - 1,2,3 [ CSSD]2013-05-23 15:55:29.426 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x22f4fa0) pid(24246) version 11:1:1:4 [ CSSD]2013-05-23 15:55:29.442 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f4fa0), client(1/0x22ff0a0) [ CSSD]2013-05-23 15:55:29.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff0a0) [ CSSD]2013-05-23 15:55:29.442 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff0a0 with con 0x22f4d90, requested num -1 [ CSSD]2013-05-23 15:55:29.442 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:29.442 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 964 (0x7f431bfdd210) [ CSSD]2013-05-23 15:55:29.443 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd210 (RPC#964) tag(3c4002c) sent to node 1 [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03c4002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014229, gin 1, member 0 [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd210 (RPC#964) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd210, ret 0, client 0x22ff0a0 member 0x22e9e40 [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff0a0 [ CSSD]2013-05-23 15:55:29.443 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 964 [ CSSD]2013-05-23 15:55:29.451 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff0a0) [ CSSD]2013-05-23 15:55:29.451 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff0a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:29.451 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e9e40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:29.451 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 965 (0x7f431bfdd290) [ CSSD]2013-05-23 15:55:29.451 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd290 (RPC#965) tag(3c5002c) sent to node 1 [ CSSD]2013-05-23 15:55:29.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03c5002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:29.451 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:29.451 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd290 (RPC#965) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:29.451 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd290, ret 0, client 0x22ff0a0 member 0x22e9e40 [ CSSD]2013-05-23 15:55:29.451 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 965 [ CSSD]2013-05-23 15:55:29.451 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e9e40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:55:29.654 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4fa0 [ CSSD]2013-05-23 15:55:29.654 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4fa0) con(0x22d7ec0) skgpid ospid 24246 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:30.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:31.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:31.252 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:31.446 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:31.446 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:55:32.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:33.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:33.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:34.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:35.311 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:35.311 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:36.449 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:36.449 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:36.875 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:55:36.875 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 966 (0x7f431bfdd310) [ CSSD]2013-05-23 15:55:36.875 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd310 (RPC#966) tag(3c6002c) sent to node 1 [ CSSD]2013-05-23 15:55:36.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:36.876 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 966 (0x7f431bfdd310) [ CSSD]2013-05-23 15:55:36.876 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd310 (RPC#966) state 6, flags 0x202 [ CSSD]2013-05-23 15:55:36.876 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd310, ret 0, client (nil) [ CSSD]2013-05-23 15:55:36.876 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 966 [ CSSD]2013-05-23 15:55:36.966 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:37.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:37.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:38.978 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:39.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:39.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:39.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:39.908 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014230, gin 1, member 0 [ CSSD]2013-05-23 15:55:39.908 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:55:39.908 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:55:39.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:39.915 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:55:39.915 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:55:40.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:41.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:41.364 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:41.453 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:41.453 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:43.074 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:43.370 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:43.370 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:44.685 [1191196992] >TRACE: clssgmAllocProc: (0x22f4fa0) allocated [ CSSD]2013-05-23 15:55:44.686 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:55:44.687 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4fa0 - 1,2,3 [ CSSD]2013-05-23 15:55:44.687 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x22f4fa0) pid(24388) version 11:1:1:4 [ CSSD]2013-05-23 15:55:44.693 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f4fa0), client(1/0x22ff0a0) [ CSSD]2013-05-23 15:55:44.693 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff0a0) [ CSSD]2013-05-23 15:55:44.693 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff0a0 with con 0x22f4d90, requested num -1 [ CSSD]2013-05-23 15:55:44.693 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:44.693 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 967 (0x7f431bfdd390) [ CSSD]2013-05-23 15:55:44.693 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd390 (RPC#967) tag(3c7002c) sent to node 1 [ CSSD]2013-05-23 15:55:44.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03c7002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:44.693 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014231, gin 1, member 0 [ CSSD]2013-05-23 15:55:44.693 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:55:44.694 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:55:44.694 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:55:44.694 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd390 (RPC#967) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:44.694 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd390, ret 0, client 0x22ff0a0 member 0x22e9e40 [ CSSD]2013-05-23 15:55:44.694 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff0a0 [ CSSD]2013-05-23 15:55:44.694 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 967 [ CSSD]2013-05-23 15:55:44.700 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff0a0) [ CSSD]2013-05-23 15:55:44.700 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff0a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:44.700 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e9e40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:44.700 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 968 (0x7f431bfdd410) [ CSSD]2013-05-23 15:55:44.701 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd410 (RPC#968) tag(3c8002c) sent to node 1 [ CSSD]2013-05-23 15:55:44.701 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03c8002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:44.701 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:44.701 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd410 (RPC#968) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:44.701 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd410, ret 0, client 0x22ff0a0 member 0x22e9e40 [ CSSD]2013-05-23 15:55:44.701 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 968 [ CSSD]2013-05-23 15:55:44.701 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e9e40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:55:44.904 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4fa0 [ CSSD]2013-05-23 15:55:44.904 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4fa0) con(0x22d7ec0) skgpid ospid 24388 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:55:45.080 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:45.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:45.421 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:46.456 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:46.456 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:46.911 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:55:46.912 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 969 (0x7f431bfdd490) [ CSSD]2013-05-23 15:55:46.912 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd490 (RPC#969) tag(3c9002c) sent to node 1 [ CSSD]2013-05-23 15:55:46.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:46.912 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 969 (0x7f431bfdd490) [ CSSD]2013-05-23 15:55:46.912 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd490 (RPC#969) state 6, flags 0x202 [ CSSD]2013-05-23 15:55:46.912 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd490, ret 0, client (nil) [ CSSD]2013-05-23 15:55:46.913 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 969 [ CSSD]2013-05-23 15:55:47.085 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:47.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:47.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:49.098 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:49.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:49.439 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:50.458 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:50.459 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:55:51.143 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:51.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:51.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:53.176 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:53.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:53.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:55.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:55.157 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014232, gin 1, member 0 [ CSSD]2013-05-23 15:55:55.157 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:55:55.157 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:55:55.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:55.163 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:55:55.163 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c135050) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:55:55.201 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:55.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:55.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:55.462 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:55:55.462 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:55:56.936 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:55:56.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 970 (0x7f431bfdd510) [ CSSD]2013-05-23 15:55:56.937 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd510 (RPC#970) tag(3ca002c) sent to node 1 [ CSSD]2013-05-23 15:55:56.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:56.937 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 970 (0x7f431bfdd510) [ CSSD]2013-05-23 15:55:56.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd510 (RPC#970) state 6, flags 0x202 [ CSSD]2013-05-23 15:55:56.937 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd510, ret 0, client (nil) [ CSSD]2013-05-23 15:55:56.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 970 [ CSSD]2013-05-23 15:55:57.253 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:57.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:57.467 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:59.259 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:55:59.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:55:59.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:55:59.945 [1191196992] >TRACE: clssgmAllocProc: (0x22f5070) allocated [ CSSD]2013-05-23 15:55:59.946 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:55:59.946 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f5070 - 1,2,3 [ CSSD]2013-05-23 15:55:59.946 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x22f5070) pid(24542) version 11:1:1:4 [ CSSD]2013-05-23 15:55:59.952 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f5070), client(1/0x22f7a80) [ CSSD]2013-05-23 15:55:59.952 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f7a80) [ CSSD]2013-05-23 15:55:59.952 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f7a80 with con 0x22d7d20, requested num -1 [ CSSD]2013-05-23 15:55:59.952 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:59.952 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 971 (0x7f431bfdd590) [ CSSD]2013-05-23 15:55:59.952 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd590 (RPC#971) tag(3cb002c) sent to node 1 [ CSSD]2013-05-23 15:55:59.952 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03cb002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:59.952 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014233, gin 1, member 0 [ CSSD]2013-05-23 15:55:59.952 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:55:59.953 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:55:59.953 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:55:59.953 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd590 (RPC#971) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:59.953 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd590, ret 0, client 0x22f7a80 member 0x22f4f50 [ CSSD]2013-05-23 15:55:59.953 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f7a80 [ CSSD]2013-05-23 15:55:59.953 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 971 [ CSSD]2013-05-23 15:55:59.959 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f7a80) [ CSSD]2013-05-23 15:55:59.959 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f7a80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:59.959 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4f50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:55:59.959 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 972 (0x7f431bfdd610) [ CSSD]2013-05-23 15:55:59.959 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd610 (RPC#972) tag(3cc002c) sent to node 1 [ CSSD]2013-05-23 15:55:59.959 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03cc002c) incarnation(174631051) [ CSSD]2013-05-23 15:55:59.960 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:55:59.960 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd610 (RPC#972) state 6, flags 0x100 [ CSSD]2013-05-23 15:55:59.960 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd610, ret 0, client 0x22f7a80 member 0x22f4f50 [ CSSD]2013-05-23 15:55:59.960 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 972 [ CSSD]2013-05-23 15:55:59.960 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4f50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:56:00.162 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f5070 [ CSSD]2013-05-23 15:56:00.162 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f5070) con(0x22d7ec0) skgpid ospid 24542 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:00.465 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:00.465 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:01.265 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:01.544 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:01.544 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:03.275 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:03.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:03.550 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:05.301 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:05.468 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:05.469 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:05.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:05.563 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:06.962 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:56:06.962 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 973 (0x7f431bfdd690) [ CSSD]2013-05-23 15:56:06.963 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd690 (RPC#973) tag(3cd002c) sent to node 1 [ CSSD]2013-05-23 15:56:06.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:06.963 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 973 (0x7f431bfdd690) [ CSSD]2013-05-23 15:56:06.963 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd690 (RPC#973) state 6, flags 0x202 [ CSSD]2013-05-23 15:56:06.963 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd690, ret 0, client (nil) [ CSSD]2013-05-23 15:56:06.964 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 973 [ CSSD]2013-05-23 15:56:07.353 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:07.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:07.568 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:09.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:09.047 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014234, gin 1, member 0 [ CSSD]2013-05-23 15:56:09.047 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:56:09.047 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:56:09.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:09.053 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:56:09.053 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:56:09.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:09.471 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:09.471 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:56:09.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:09.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:10.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:10.406 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014235, gin 1, member 0 [ CSSD]2013-05-23 15:56:10.406 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:56:10.406 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:56:10.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:10.412 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:56:10.412 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c109eb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:56:11.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:11.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:11.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:12.814 [1191196992] >TRACE: clssgmAllocProc: (0x22f5070) allocated [ CSSD]2013-05-23 15:56:12.815 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:56:12.816 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f5070 - 1,2,3 [ CSSD]2013-05-23 15:56:12.816 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x22f5070) pid(24672) version 11:1:1:4 [ CSSD]2013-05-23 15:56:12.833 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f5070), client(1/0x22f7a80) [ CSSD]2013-05-23 15:56:12.833 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f7a80) [ CSSD]2013-05-23 15:56:12.833 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f7a80 with con 0x22d7d20, requested num -1 [ CSSD]2013-05-23 15:56:12.833 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:12.833 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 974 (0x7f431bfdd710) [ CSSD]2013-05-23 15:56:12.833 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd710 (RPC#974) tag(3ce002c) sent to node 1 [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ce002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014236, gin 1, member 0 [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd710 (RPC#974) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd710, ret 0, client 0x22f7a80 member 0x22f4f50 [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f7a80 [ CSSD]2013-05-23 15:56:12.834 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 974 [ CSSD]2013-05-23 15:56:12.841 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f7a80) [ CSSD]2013-05-23 15:56:12.841 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f7a80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:12.841 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4f50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:12.841 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 975 (0x7f431bfdd790) [ CSSD]2013-05-23 15:56:12.841 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd790 (RPC#975) tag(3cf002c) sent to node 1 [ CSSD]2013-05-23 15:56:12.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03cf002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:12.841 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:12.841 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd790 (RPC#975) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:12.841 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd790, ret 0, client 0x22f7a80 member 0x22f4f50 [ CSSD]2013-05-23 15:56:12.841 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 975 [ CSSD]2013-05-23 15:56:12.841 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4f50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:56:13.044 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f5070 [ CSSD]2013-05-23 15:56:13.044 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f5070) con(0x22d7ec0) skgpid ospid 24672 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:13.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:13.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:13.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:14.474 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:14.474 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:15.196 [1191196992] >TRACE: clssgmAllocProc: (0x22f5070) allocated [ CSSD]2013-05-23 15:56:15.197 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:56:15.198 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f5070 - 1,2,3 [ CSSD]2013-05-23 15:56:15.198 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x22f5070) pid(24730) version 11:1:1:4 [ CSSD]2013-05-23 15:56:15.214 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f5070), client(1/0x22f7a80) [ CSSD]2013-05-23 15:56:15.214 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f7a80) [ CSSD]2013-05-23 15:56:15.214 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f7a80 with con 0x22d7d20, requested num -1 [ CSSD]2013-05-23 15:56:15.214 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:15.214 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 976 (0x7f431bfdd810) [ CSSD]2013-05-23 15:56:15.214 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd810 (RPC#976) tag(3d0002c) sent to node 1 [ CSSD]2013-05-23 15:56:15.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d0002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:15.214 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014237, gin 1, member 0 [ CSSD]2013-05-23 15:56:15.214 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:56:15.215 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:56:15.215 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:56:15.215 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd810 (RPC#976) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:15.215 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd810, ret 0, client 0x22f7a80 member 0x22f4f50 [ CSSD]2013-05-23 15:56:15.215 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f7a80 [ CSSD]2013-05-23 15:56:15.215 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 976 [ CSSD]2013-05-23 15:56:15.221 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f7a80) [ CSSD]2013-05-23 15:56:15.221 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f7a80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:15.221 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4f50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:15.221 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 977 (0x7f431bfdd890) [ CSSD]2013-05-23 15:56:15.221 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd890 (RPC#977) tag(3d1002c) sent to node 1 [ CSSD]2013-05-23 15:56:15.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03d1002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:15.222 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:15.222 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd890 (RPC#977) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:15.222 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd890, ret 0, client 0x22f7a80 member 0x22f4f50 [ CSSD]2013-05-23 15:56:15.222 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 977 [ CSSD]2013-05-23 15:56:15.222 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4f50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:56:15.381 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:15.424 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f5070 [ CSSD]2013-05-23 15:56:15.424 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f5070) con(0x22d7ec0) skgpid ospid 24730 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:15.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:15.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:15.695 [1191196992] >TRACE: clssgmAllocProc: (0x22f4fa0) allocated [ CSSD]2013-05-23 15:56:15.696 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:56:15.697 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4fa0 - 1,2,3 [ CSSD]2013-05-23 15:56:15.697 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x22f4fa0) pid(24771) version 11:1:1:4 [ CSSD]2013-05-23 15:56:15.703 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f4fa0), client(1/0x22ff0a0) [ CSSD]2013-05-23 15:56:15.703 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff0a0) [ CSSD]2013-05-23 15:56:15.703 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22ff0a0 with con 0x22f4d90, requested num -1 [ CSSD]2013-05-23 15:56:15.703 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:56:15.703 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 978 (0x7f431bfdd910) [ CSSD]2013-05-23 15:56:15.704 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd910 (RPC#978) tag(3d2002c) sent to node 1 [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d2002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014238, gin 1, member 0 [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9b40/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd910 (RPC#978) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd910, ret 0, client 0x22ff0a0 member 0x22e9e40 [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff0a0 [ CSSD]2013-05-23 15:56:15.704 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 978 [ CSSD]2013-05-23 15:56:15.711 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff0a0) [ CSSD]2013-05-23 15:56:15.711 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff0a0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:56:15.711 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e9e40) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 15:56:15.711 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 979 (0x7f431bfdd990) [ CSSD]2013-05-23 15:56:15.711 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd990 (RPC#979) tag(3d3002c) sent to node 1 [ CSSD]2013-05-23 15:56:15.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03d3002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:15.711 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 15:56:15.711 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd990 (RPC#979) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:15.711 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd990, ret 0, client 0x22ff0a0 member 0x22e9e40 [ CSSD]2013-05-23 15:56:15.711 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 979 [ CSSD]2013-05-23 15:56:15.711 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22e9e40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:56:15.818 [1191196992] >TRACE: clssgmAllocProc: (0x2327d10) allocated [ CSSD]2013-05-23 15:56:15.819 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:56:15.820 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327d10 - 1,2,3 [ CSSD]2013-05-23 15:56:15.820 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9ac0) proc(0x2327d10) pid(24776) version 11:1:1:4 [ CSSD]2013-05-23 15:56:15.844 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:56:15.846 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:56:15.849 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327d10 [ CSSD]2013-05-23 15:56:15.849 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327d10) con(0x22e9ac0) skgpid ospid 24776 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:15.914 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4fa0 [ CSSD]2013-05-23 15:56:15.914 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4fa0) con(0x22d7ec0) skgpid ospid 24771 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:16.988 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:56:16.988 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 980 (0x7f431bfdda10) [ CSSD]2013-05-23 15:56:16.989 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdda10 (RPC#980) tag(3d4002c) sent to node 1 [ CSSD]2013-05-23 15:56:16.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:16.989 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 980 (0x7f431bfdda10) [ CSSD]2013-05-23 15:56:16.989 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdda10 (RPC#980) state 6, flags 0x202 [ CSSD]2013-05-23 15:56:16.989 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdda10, ret 0, client (nil) [ CSSD]2013-05-23 15:56:16.989 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 980 [ CSSD]2013-05-23 15:56:17.394 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:17.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:17.668 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:19.399 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:19.477 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:19.477 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:19.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:19.739 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:21.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:21.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:21.744 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:23.088 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f1e60) [ CSSD]2013-05-23 15:56:23.088 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f1e60), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:56:23.088 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f1e60), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:56:23.088 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 15:56:23.093 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x231ee10) [ CSSD]2013-05-23 15:56:23.093 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x231ee10), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:56:23.093 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x231ee10), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:56:23.093 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 15:56:23.093 [1191196992] >TRACE: clssgmDeadProc: proc 0x2324180 [ CSSD]2013-05-23 15:56:23.093 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2324180) con(0x23233c0) skgpid 19451 ospid 19451 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:23.457 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:23.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:23.757 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:24.480 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:24.480 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:25.462 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:25.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:25.666 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014239, gin 1, member 0 [ CSSD]2013-05-23 15:56:25.666 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:56:25.666 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:56:25.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:25.673 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:56:25.673 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e6d60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:56:25.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:25.763 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:27.012 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:56:27.012 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 981 (0x7f431bfdda90) [ CSSD]2013-05-23 15:56:27.013 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdda90 (RPC#981) tag(3d5002c) sent to node 1 [ CSSD]2013-05-23 15:56:27.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:27.013 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 981 (0x7f431bfdda90) [ CSSD]2013-05-23 15:56:27.013 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdda90 (RPC#981) state 6, flags 0x202 [ CSSD]2013-05-23 15:56:27.013 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdda90, ret 0, client (nil) [ CSSD]2013-05-23 15:56:27.013 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 981 [ CSSD]2013-05-23 15:56:27.475 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:27.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:27.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:29.480 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:29.483 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:29.483 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:29.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:29.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:30.455 [1191196992] >TRACE: clssgmAllocProc: (0x2323da0) allocated [ CSSD]2013-05-23 15:56:30.456 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:56:30.456 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323da0 - 1,2,3 [ CSSD]2013-05-23 15:56:30.456 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x2323da0) pid(24918) version 11:1:1:4 [ CSSD]2013-05-23 15:56:30.462 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2323da0), client(1/0x2324050) [ CSSD]2013-05-23 15:56:30.462 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:56:30.462 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2324050 with con 0x2323fa0, requested num -1 [ CSSD]2013-05-23 15:56:30.462 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:30.462 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 982 (0x7f431bfddb10) [ CSSD]2013-05-23 15:56:30.462 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddb10 (RPC#982) tag(3d6002c) sent to node 1 [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d6002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014240, gin 1, member 0 [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f81c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddb10 (RPC#982) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfddb10, ret 0, client 0x2324050 member 0x22fef00 [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2324050 [ CSSD]2013-05-23 15:56:30.463 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 982 [ CSSD]2013-05-23 15:56:30.469 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:56:30.469 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2324050), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:30.469 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fef00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:30.469 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 983 (0x7f431bfddb90) [ CSSD]2013-05-23 15:56:30.469 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddb90 (RPC#983) tag(3d7002c) sent to node 1 [ CSSD]2013-05-23 15:56:30.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03d7002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:30.470 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:30.470 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddb90 (RPC#983) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:30.470 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfddb90, ret 0, client 0x2324050 member 0x22fef00 [ CSSD]2013-05-23 15:56:30.470 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 983 [ CSSD]2013-05-23 15:56:30.470 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fef00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:56:30.672 [1191196992] >TRACE: clssgmDeadProc: proc 0x2323da0 [ CSSD]2013-05-23 15:56:30.672 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2323da0) con(0x22d7ec0) skgpid ospid 24918 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:31.505 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:31.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:31.846 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:33.485 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:33.485 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:56:33.558 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:33.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:33.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:35.564 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:35.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:35.983 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:37.043 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:56:37.043 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 984 (0x7f431bfddc10) [ CSSD]2013-05-23 15:56:37.043 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddc10 (RPC#984) tag(3d8002c) sent to node 1 [ CSSD]2013-05-23 15:56:37.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:37.044 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 984 (0x7f431bfddc10) [ CSSD]2013-05-23 15:56:37.044 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddc10 (RPC#984) state 6, flags 0x202 [ CSSD]2013-05-23 15:56:37.044 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfddc10, ret 0, client (nil) [ CSSD]2013-05-23 15:56:37.044 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 984 [ CSSD]2013-05-23 15:56:37.565 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:37.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:37.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:38.488 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:38.488 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:39.571 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:40.022 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:40.022 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:40.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:40.916 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014241, gin 1, member 0 [ CSSD]2013-05-23 15:56:40.916 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:56:40.916 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:56:40.923 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:40.923 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:56:40.923 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:56:41.576 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:42.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:42.028 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:43.491 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:43.492 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:43.588 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:44.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:44.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:45.614 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:45.704 [1191196992] >TRACE: clssgmAllocProc: (0x2323fa0) allocated [ CSSD]2013-05-23 15:56:45.705 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:56:45.706 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323fa0 - 1,2,3 [ CSSD]2013-05-23 15:56:45.706 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302580) proc(0x2323fa0) pid(25076) version 11:1:1:4 [ CSSD]2013-05-23 15:56:45.711 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2323fa0), client(1/0x22d7c40) [ CSSD]2013-05-23 15:56:45.711 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7c40) [ CSSD]2013-05-23 15:56:45.711 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7c40 with con 0x22d7940, requested num -1 [ CSSD]2013-05-23 15:56:45.711 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:45.711 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 985 (0x7f431bfddc90) [ CSSD]2013-05-23 15:56:45.711 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddc90 (RPC#985) tag(3d9002c) sent to node 1 [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d9002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014242, gin 1, member 0 [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f1730/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddc90 (RPC#985) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfddc90, ret 0, client 0x22d7c40 member 0x2323e90 [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7c40 [ CSSD]2013-05-23 15:56:45.712 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 985 [ CSSD]2013-05-23 15:56:45.718 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7c40) [ CSSD]2013-05-23 15:56:45.718 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7c40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:45.718 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2323e90) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:56:45.718 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 986 (0x7f431bfddd10) [ CSSD]2013-05-23 15:56:45.718 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddd10 (RPC#986) tag(3da002c) sent to node 1 [ CSSD]2013-05-23 15:56:45.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03da002c) incarnation(174631051) [ CSSD]2013-05-23 15:56:45.719 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:56:45.719 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddd10 (RPC#986) state 6, flags 0x100 [ CSSD]2013-05-23 15:56:45.719 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfddd10, ret 0, client 0x22d7c40 member 0x2323e90 [ CSSD]2013-05-23 15:56:45.719 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 986 [ CSSD]2013-05-23 15:56:45.719 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2323e90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:56:45.921 [1191196992] >TRACE: clssgmDeadProc: proc 0x2323fa0 [ CSSD]2013-05-23 15:56:45.921 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2323fa0) con(0x2302580) skgpid ospid 25076 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:56:46.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:46.045 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:47.068 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:56:47.068 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 987 (0x7f431bfddd90) [ CSSD]2013-05-23 15:56:47.069 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddd90 (RPC#987) tag(3db002c) sent to node 1 [ CSSD]2013-05-23 15:56:47.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:47.069 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 987 (0x7f431bfddd90) [ CSSD]2013-05-23 15:56:47.069 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddd90 (RPC#987) state 6, flags 0x202 [ CSSD]2013-05-23 15:56:47.069 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfddd90, ret 0, client (nil) [ CSSD]2013-05-23 15:56:47.069 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 987 [ CSSD]2013-05-23 15:56:47.685 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:48.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:48.050 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:48.495 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:48.495 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:49.690 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:50.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:50.061 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:51.695 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:52.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:52.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:52.497 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:52.497 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:56:53.708 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:54.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:54.120 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:55.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:56.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:56.126 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:56.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:56.166 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014243, gin 1, member 0 [ CSSD]2013-05-23 15:56:56.166 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:56:56.166 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:56:56.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:56.174 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:56:56.174 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e6d60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:56:57.091 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:56:57.092 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 988 (0x7f431bfdde10) [ CSSD]2013-05-23 15:56:57.092 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdde10 (RPC#988) tag(3dc002c) sent to node 1 [ CSSD]2013-05-23 15:56:57.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:57.092 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 988 (0x7f431bfdde10) [ CSSD]2013-05-23 15:56:57.092 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdde10 (RPC#988) state 6, flags 0x202 [ CSSD]2013-05-23 15:56:57.092 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdde10, ret 0, client (nil) [ CSSD]2013-05-23 15:56:57.093 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 988 [ CSSD]2013-05-23 15:56:57.500 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:56:57.500 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:56:57.785 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:56:58.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:56:58.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:56:59.791 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:00.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:00.161 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:00.954 [1191196992] >TRACE: clssgmAllocProc: (0x2323da0) allocated [ CSSD]2013-05-23 15:57:00.955 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:57:00.956 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323da0 - 1,2,3 [ CSSD]2013-05-23 15:57:00.956 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x2323da0) pid(25222) version 11:1:1:4 [ CSSD]2013-05-23 15:57:00.972 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2323da0), client(1/0x2324050) [ CSSD]2013-05-23 15:57:00.972 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:57:00.972 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2324050 with con 0x2323fa0, requested num -1 [ CSSD]2013-05-23 15:57:00.972 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:00.972 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 989 (0x7f431bfdde90) [ CSSD]2013-05-23 15:57:00.973 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdde90 (RPC#989) tag(3dd002c) sent to node 1 [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03dd002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014244, gin 1, member 0 [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f81c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdde90 (RPC#989) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdde90, ret 0, client 0x2324050 member 0x22fef00 [ CSSD]2013-05-23 15:57:00.973 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2324050 [ CSSD]2013-05-23 15:57:00.974 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 989 [ CSSD]2013-05-23 15:57:00.980 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:57:00.980 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2324050), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:00.980 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fef00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:00.980 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 990 (0x7f431bfddf10) [ CSSD]2013-05-23 15:57:00.980 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddf10 (RPC#990) tag(3de002c) sent to node 1 [ CSSD]2013-05-23 15:57:00.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03de002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:00.981 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:00.981 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddf10 (RPC#990) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:00.981 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfddf10, ret 0, client 0x2324050 member 0x22fef00 [ CSSD]2013-05-23 15:57:00.981 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 990 [ CSSD]2013-05-23 15:57:00.981 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fef00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:57:01.183 [1191196992] >TRACE: clssgmDeadProc: proc 0x2323da0 [ CSSD]2013-05-23 15:57:01.183 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2323da0) con(0x22d7ec0) skgpid ospid 25222 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:01.797 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:02.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:02.167 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:02.503 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:02.503 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:03.808 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:04.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:04.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:05.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:06.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:06.196 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:07.117 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:57:07.117 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 991 (0x7f431bfddf90) [ CSSD]2013-05-23 15:57:07.118 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddf90 (RPC#991) tag(3df002c) sent to node 1 [ CSSD]2013-05-23 15:57:07.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:07.118 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 991 (0x7f431bfddf90) [ CSSD]2013-05-23 15:57:07.118 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddf90 (RPC#991) state 6, flags 0x202 [ CSSD]2013-05-23 15:57:07.118 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfddf90, ret 0, client (nil) [ CSSD]2013-05-23 15:57:07.118 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 991 [ CSSD]2013-05-23 15:57:07.506 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:07.506 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:07.866 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:08.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:08.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:09.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:09.307 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014245, gin 1, member 0 [ CSSD]2013-05-23 15:57:09.307 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:09.307 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:09.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:09.314 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:09.314 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:09.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:10.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:10.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:11.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:11.197 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014246, gin 1, member 0 [ CSSD]2013-05-23 15:57:11.197 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:11.197 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:11.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:11.202 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:11.202 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c135050) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:11.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:11.429 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014247, gin 1, member 0 [ CSSD]2013-05-23 15:57:11.429 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:11.429 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:11.436 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:11.436 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:11.436 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:11.898 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:12.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:12.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:12.509 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:12.510 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:13.085 [1191196992] >TRACE: clssgmAllocProc: (0x2323da0) allocated [ CSSD]2013-05-23 15:57:13.086 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:57:13.087 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323da0 - 1,2,3 [ CSSD]2013-05-23 15:57:13.087 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x2323da0) pid(25439) version 11:1:1:4 [ CSSD]2013-05-23 15:57:13.092 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2323da0), client(1/0x2324050) [ CSSD]2013-05-23 15:57:13.092 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:57:13.092 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2324050 with con 0x2323fa0, requested num -1 [ CSSD]2013-05-23 15:57:13.092 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:13.093 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 992 (0x7f431bfde010) [ CSSD]2013-05-23 15:57:13.093 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde010 (RPC#992) tag(3e0002c) sent to node 1 [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e0002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014248, gin 1, member 0 [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f81c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde010 (RPC#992) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde010, ret 0, client 0x2324050 member 0x22fef00 [ CSSD]2013-05-23 15:57:13.093 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2324050 [ CSSD]2013-05-23 15:57:13.094 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 992 [ CSSD]2013-05-23 15:57:13.100 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:57:13.100 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2324050), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:13.100 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fef00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:13.100 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 993 (0x7f431bfde090) [ CSSD]2013-05-23 15:57:13.100 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde090 (RPC#993) tag(3e1002c) sent to node 1 [ CSSD]2013-05-23 15:57:13.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e1002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:13.100 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:13.100 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde090 (RPC#993) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:13.100 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde090, ret 0, client 0x2324050 member 0x22fef00 [ CSSD]2013-05-23 15:57:13.101 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 993 [ CSSD]2013-05-23 15:57:13.101 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fef00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:57:13.303 [1191196992] >TRACE: clssgmDeadProc: proc 0x2323da0 [ CSSD]2013-05-23 15:57:13.303 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2323da0) con(0x22d7ec0) skgpid ospid 25439 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:13.989 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:14.070 [1191196992] >TRACE: clssgmAllocProc: (0x2323fa0) allocated [ CSSD]2013-05-23 15:57:14.072 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:57:14.073 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323fa0 - 1,2,3 [ CSSD]2013-05-23 15:57:14.073 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f7e40) proc(0x2323fa0) pid(25533) version 11:1:1:4 [ CSSD]2013-05-23 15:57:14.213 [1191196992] >TRACE: clssgmAllocProc: (0x22f4d90) allocated [ CSSD]2013-05-23 15:57:14.214 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:57:14.214 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4d90 - 1,2,3 [ CSSD]2013-05-23 15:57:14.214 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f81d0) proc(0x22f4d90) pid(25544) version 11:1:1:4 [ CSSD]2013-05-23 15:57:14.233 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:57:14.235 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:57:14.238 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4d90 [ CSSD]2013-05-23 15:57:14.238 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4d90) con(0x22f81d0) skgpid ospid 25544 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:14.310 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(62/0x22f4f10) [ CSSD]2013-05-23 15:57:14.310 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 62 (0x22f4f10) [ CSSD]2013-05-23 15:57:14.310 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22f4f10 with con 0x22f80f0, requested num -1 [ CSSD]2013-05-23 15:57:14.310 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:57:14.310 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 994 (0x7f431bfde110) [ CSSD]2013-05-23 15:57:14.311 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde110 (RPC#994) tag(3e2002c) sent to node 1 [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e2002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014249, gin 1, member 0 [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x22f8280/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde110 (RPC#994) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde110, ret 0, client 0x22f4f10 member 0x22d7940 [ CSSD]2013-05-23 15:57:14.311 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4f10 [ CSSD]2013-05-23 15:57:14.312 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 994 [ CSSD]2013-05-23 15:57:14.319 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 62 (0x22f4f10) [ CSSD]2013-05-23 15:57:14.319 [1191196992] >TRACE: clssgmExitGrock: client 62 (0x22f4f10), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:57:14.319 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7940) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 15:57:14.319 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 995 (0x7f431bfde190) [ CSSD]2013-05-23 15:57:14.319 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde190 (RPC#995) tag(3e3002c) sent to node 1 [ CSSD]2013-05-23 15:57:14.319 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e3002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:14.319 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 15:57:14.319 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde190 (RPC#995) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:14.319 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde190, ret 0, client 0x22f4f10 member 0x22d7940 [ CSSD]2013-05-23 15:57:14.320 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 995 [ CSSD]2013-05-23 15:57:14.320 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22d7940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:57:14.321 [1191196992] >TRACE: clssgmDeadProc: proc 0x2323fa0 [ CSSD]2013-05-23 15:57:14.321 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2323fa0) con(0x22f7e40) skgpid ospid 25533 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:14.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:14.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:16.014 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:16.212 [1191196992] >TRACE: clssgmAllocProc: (0x2323fa0) allocated [ CSSD]2013-05-23 15:57:16.213 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:57:16.213 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323fa0 - 1,2,3 [ CSSD]2013-05-23 15:57:16.213 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1e90) proc(0x2323fa0) pid(25593) version 11:1:1:4 [ CSSD]2013-05-23 15:57:16.219 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2323fa0), client(1/0x22d7c40) [ CSSD]2013-05-23 15:57:16.219 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7c40) [ CSSD]2013-05-23 15:57:16.219 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7c40 with con 0x22d7940, requested num -1 [ CSSD]2013-05-23 15:57:16.219 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:16.219 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 996 (0x7f431bfde210) [ CSSD]2013-05-23 15:57:16.219 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde210 (RPC#996) tag(3e4002c) sent to node 1 [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e4002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014250, gin 1, member 0 [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f1730/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde210 (RPC#996) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde210, ret 0, client 0x22d7c40 member 0x2323e90 [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7c40 [ CSSD]2013-05-23 15:57:16.220 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 996 [ CSSD]2013-05-23 15:57:16.226 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7c40) [ CSSD]2013-05-23 15:57:16.226 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7c40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:16.226 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2323e90) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:16.226 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 997 (0x7f431bfde290) [ CSSD]2013-05-23 15:57:16.226 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde290 (RPC#997) tag(3e5002c) sent to node 1 [ CSSD]2013-05-23 15:57:16.227 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e5002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:16.227 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:16.227 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde290 (RPC#997) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:16.227 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde290, ret 0, client 0x22d7c40 member 0x2323e90 [ CSSD]2013-05-23 15:57:16.227 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 997 [ CSSD]2013-05-23 15:57:16.227 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2323e90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:57:16.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:16.375 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:16.429 [1191196992] >TRACE: clssgmDeadProc: proc 0x2323fa0 [ CSSD]2013-05-23 15:57:16.429 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2323fa0) con(0x22f1e90) skgpid ospid 25593 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:17.141 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:57:17.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 998 (0x7f431bfde310) [ CSSD]2013-05-23 15:57:17.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde310 (RPC#998) tag(3e6002c) sent to node 1 [ CSSD]2013-05-23 15:57:17.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:17.142 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 998 (0x7f431bfde310) [ CSSD]2013-05-23 15:57:17.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde310 (RPC#998) state 6, flags 0x202 [ CSSD]2013-05-23 15:57:17.142 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde310, ret 0, client (nil) [ CSSD]2013-05-23 15:57:17.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 998 [ CSSD]2013-05-23 15:57:17.513 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:17.513 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:18.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:18.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:18.381 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:20.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:20.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:20.452 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:22.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:22.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:22.458 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:22.516 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:22.516 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:22.983 [1191196992] >TRACE: clssgmAllocProc: (0x2323da0) allocated [ CSSD]2013-05-23 15:57:22.984 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:57:22.985 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2323da0 - 1,2,3 [ CSSD]2013-05-23 15:57:22.985 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7ec0) proc(0x2323da0) pid(25691/25691) version 11:1:1:4 [ CSSD]2013-05-23 15:57:23.001 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2323da0), client(1/0x2324050) [ CSSD]2013-05-23 15:57:23.001 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2324050) [ CSSD]2013-05-23 15:57:23.001 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:57:23.001 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 15:57:24.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:24.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:24.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:26.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:26.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:26.477 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:26.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:26.693 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014251, gin 1, member 0 [ CSSD]2013-05-23 15:57:26.693 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:26.693 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:26.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:26.700 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:26.700 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c109eb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:27.165 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:57:27.165 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 999 (0x7f431bfde390) [ CSSD]2013-05-23 15:57:27.165 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde390 (RPC#999) tag(3e7002c) sent to node 1 [ CSSD]2013-05-23 15:57:27.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:27.166 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 999 (0x7f431bfde390) [ CSSD]2013-05-23 15:57:27.166 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde390 (RPC#999) state 6, flags 0x202 [ CSSD]2013-05-23 15:57:27.166 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde390, ret 0, client (nil) [ CSSD]2013-05-23 15:57:27.166 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 999 [ CSSD]2013-05-23 15:57:27.519 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:27.520 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:28.168 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:28.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:28.482 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:30.173 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:30.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:30.535 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:31.461 [1191196992] >TRACE: clssgmAllocProc: (0x22d7ac0) allocated [ CSSD]2013-05-23 15:57:31.462 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:57:31.463 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7ac0 - 1,2,3 [ CSSD]2013-05-23 15:57:31.463 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7cd0) proc(0x22d7ac0) pid(25748) version 11:1:1:4 [ CSSD]2013-05-23 15:57:31.469 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22d7ac0), client(1/0x231ed00) [ CSSD]2013-05-23 15:57:31.469 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231ed00) [ CSSD]2013-05-23 15:57:31.469 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231ed00 with con 0x231e7f0, requested num -1 [ CSSD]2013-05-23 15:57:31.469 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:31.469 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1000 (0x7f431bfde410) [ CSSD]2013-05-23 15:57:31.469 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde410 (RPC#1000) tag(3e8002c) sent to node 1 [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e8002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014252, gin 1, member 0 [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e96e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde410 (RPC#1000) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:31.469 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde410, ret 0, client 0x231ed00 member 0x22f5130 [ CSSD]2013-05-23 15:57:31.470 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231ed00 [ CSSD]2013-05-23 15:57:31.470 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1000 [ CSSD]2013-05-23 15:57:31.476 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231ed00) [ CSSD]2013-05-23 15:57:31.476 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231ed00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:31.476 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f5130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:31.476 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1001 (0x7f431bfde490) [ CSSD]2013-05-23 15:57:31.476 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde490 (RPC#1001) tag(3e9002c) sent to node 1 [ CSSD]2013-05-23 15:57:31.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e9002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:31.477 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:31.477 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde490 (RPC#1001) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:31.477 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde490, ret 0, client 0x231ed00 member 0x22f5130 [ CSSD]2013-05-23 15:57:31.477 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1001 [ CSSD]2013-05-23 15:57:31.477 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f5130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:57:31.679 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7ac0 [ CSSD]2013-05-23 15:57:31.679 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7ac0) con(0x22d7cd0) skgpid ospid 25748 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:32.185 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:32.523 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:32.523 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:32.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:32.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:34.191 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:34.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:34.552 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:36.197 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:36.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:36.557 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:37.188 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:57:37.188 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1002 (0x7f431bfde510) [ CSSD]2013-05-23 15:57:37.188 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde510 (RPC#1002) tag(3ea002c) sent to node 1 [ CSSD]2013-05-23 15:57:37.189 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:37.189 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1002 (0x7f431bfde510) [ CSSD]2013-05-23 15:57:37.189 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde510 (RPC#1002) state 6, flags 0x202 [ CSSD]2013-05-23 15:57:37.189 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde510, ret 0, client (nil) [ CSSD]2013-05-23 15:57:37.189 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1002 [ CSSD]2013-05-23 15:57:37.525 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:37.526 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:38.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:38.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:38.563 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:40.213 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:40.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:40.574 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:40.720 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:40.720 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014253, gin 1, member 0 [ CSSD]2013-05-23 15:57:40.720 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:40.720 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:40.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:40.727 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:40.727 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:41.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:41.957 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014254, gin 1, member 0 [ CSSD]2013-05-23 15:57:41.957 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:41.957 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:41.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:41.963 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:41.963 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e6d60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:42.266 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:42.529 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:42.529 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:42.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:42.600 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:44.272 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:44.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:44.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:46.278 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:46.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:46.638 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:46.711 [1191196992] >TRACE: clssgmAllocProc: (0x22d7ac0) allocated [ CSSD]2013-05-23 15:57:46.713 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:57:46.713 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7ac0 - 1,2,3 [ CSSD]2013-05-23 15:57:46.713 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7cd0) proc(0x22d7ac0) pid(25944) version 11:1:1:4 [ CSSD]2013-05-23 15:57:46.730 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22d7ac0), client(1/0x231ed00) [ CSSD]2013-05-23 15:57:46.730 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231ed00) [ CSSD]2013-05-23 15:57:46.730 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231ed00 with con 0x231e7f0, requested num -1 [ CSSD]2013-05-23 15:57:46.730 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:46.730 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1003 (0x7f431bfde590) [ CSSD]2013-05-23 15:57:46.730 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde590 (RPC#1003) tag(3eb002c) sent to node 1 [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03eb002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014255, gin 1, member 0 [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e96e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde590 (RPC#1003) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde590, ret 0, client 0x231ed00 member 0x22f5130 [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231ed00 [ CSSD]2013-05-23 15:57:46.731 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1003 [ CSSD]2013-05-23 15:57:46.738 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231ed00) [ CSSD]2013-05-23 15:57:46.738 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231ed00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:46.738 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f5130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:57:46.738 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1004 (0x7f431bfde610) [ CSSD]2013-05-23 15:57:46.738 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde610 (RPC#1004) tag(3ec002c) sent to node 1 [ CSSD]2013-05-23 15:57:46.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ec002c) incarnation(174631051) [ CSSD]2013-05-23 15:57:46.739 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:57:46.739 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde610 (RPC#1004) state 6, flags 0x100 [ CSSD]2013-05-23 15:57:46.739 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde610, ret 0, client 0x231ed00 member 0x22f5130 [ CSSD]2013-05-23 15:57:46.739 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1004 [ CSSD]2013-05-23 15:57:46.739 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f5130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:57:46.941 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7ac0 [ CSSD]2013-05-23 15:57:46.941 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7ac0) con(0x22d7cd0) skgpid ospid 25944 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:57:47.220 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:57:47.221 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1005 (0x7f431bfde690) [ CSSD]2013-05-23 15:57:47.221 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde690 (RPC#1005) tag(3ed002c) sent to node 1 [ CSSD]2013-05-23 15:57:47.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:47.222 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1005 (0x7f431bfde690) [ CSSD]2013-05-23 15:57:47.222 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde690 (RPC#1005) state 6, flags 0x202 [ CSSD]2013-05-23 15:57:47.222 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde690, ret 0, client (nil) [ CSSD]2013-05-23 15:57:47.222 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1005 [ CSSD]2013-05-23 15:57:47.532 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:47.532 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:48.307 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:48.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:48.649 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:50.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:50.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:50.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:52.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:52.535 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:52.536 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:57:52.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:52.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:54.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:54.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:54.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:56.378 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:56.538 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:57:56.539 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:57:56.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:56.738 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:57:57.205 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:57.205 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014256, gin 1, member 0 [ CSSD]2013-05-23 15:57:57.205 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:57:57.206 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:57:57.212 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:57.212 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:57:57.212 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:57:57.246 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:57:57.247 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1006 (0x7f431bfde710) [ CSSD]2013-05-23 15:57:57.247 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde710 (RPC#1006) tag(3ee002c) sent to node 1 [ CSSD]2013-05-23 15:57:57.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:57.247 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1006 (0x7f431bfde710) [ CSSD]2013-05-23 15:57:57.247 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde710 (RPC#1006) state 6, flags 0x202 [ CSSD]2013-05-23 15:57:57.247 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde710, ret 0, client (nil) [ CSSD]2013-05-23 15:57:57.248 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1006 [ CSSD]2013-05-23 15:57:58.406 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:57:58.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:57:58.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:00.413 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:00.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:00.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:01.542 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:01.542 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:01.974 [1191196992] >TRACE: clssgmAllocProc: (0x22d7ac0) allocated [ CSSD]2013-05-23 15:58:01.975 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:58:01.975 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7ac0 - 1,2,3 [ CSSD]2013-05-23 15:58:01.975 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7cd0) proc(0x22d7ac0) pid(26120) version 11:1:1:4 [ CSSD]2013-05-23 15:58:01.982 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22d7ac0), client(1/0x231ed00) [ CSSD]2013-05-23 15:58:01.982 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231ed00) [ CSSD]2013-05-23 15:58:01.982 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231ed00 with con 0x231e7f0, requested num -1 [ CSSD]2013-05-23 15:58:01.982 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:01.982 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1007 (0x7f431bfde790) [ CSSD]2013-05-23 15:58:01.982 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde790 (RPC#1007) tag(3ef002c) sent to node 1 [ CSSD]2013-05-23 15:58:01.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ef002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:01.982 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014257, gin 1, member 0 [ CSSD]2013-05-23 15:58:01.982 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e96e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:58:01.983 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:58:01.983 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:58:01.983 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde790 (RPC#1007) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:01.983 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde790, ret 0, client 0x231ed00 member 0x22f5130 [ CSSD]2013-05-23 15:58:01.983 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231ed00 [ CSSD]2013-05-23 15:58:01.983 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1007 [ CSSD]2013-05-23 15:58:01.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231ed00) [ CSSD]2013-05-23 15:58:01.989 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231ed00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:01.989 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f5130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:01.989 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1008 (0x7f431bfde810) [ CSSD]2013-05-23 15:58:01.989 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde810 (RPC#1008) tag(3f0002c) sent to node 1 [ CSSD]2013-05-23 15:58:01.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f0002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:01.990 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:01.990 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde810 (RPC#1008) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:01.990 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde810, ret 0, client 0x231ed00 member 0x22f5130 [ CSSD]2013-05-23 15:58:01.990 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1008 [ CSSD]2013-05-23 15:58:01.990 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f5130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:58:02.192 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7ac0 [ CSSD]2013-05-23 15:58:02.192 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7ac0) con(0x22d7cd0) skgpid ospid 26120 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:02.416 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:02.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:02.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:04.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:04.833 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:04.833 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:06.448 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:06.545 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:06.545 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:06.839 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:06.839 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:07.272 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:58:07.272 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1009 (0x7f431bfde890) [ CSSD]2013-05-23 15:58:07.272 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde890 (RPC#1009) tag(3f1002c) sent to node 1 [ CSSD]2013-05-23 15:58:07.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:07.273 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1009 (0x7f431bfde890) [ CSSD]2013-05-23 15:58:07.273 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde890 (RPC#1009) state 6, flags 0x202 [ CSSD]2013-05-23 15:58:07.273 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde890, ret 0, client (nil) [ CSSD]2013-05-23 15:58:07.273 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1009 [ CSSD]2013-05-23 15:58:08.500 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:08.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:08.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:09.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:09.575 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014258, gin 1, member 0 [ CSSD]2013-05-23 15:58:09.575 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f18e0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:58:09.575 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:58:09.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:09.582 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:58:09.582 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7a00) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:58:10.506 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:10.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:10.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:11.548 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:11.548 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:12.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:12.472 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014259, gin 1, member 0 [ CSSD]2013-05-23 15:58:12.472 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f18e0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:58:12.472 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:58:12.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:12.478 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:58:12.478 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:58:12.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:12.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:12.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:13.343 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 15:58:13.344 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:58:13.345 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 15:58:13.345 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(26237) version 11:1:1:4 [ CSSD]2013-05-23 15:58:13.351 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 15:58:13.351 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 15:58:13.351 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:58:13.351 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:13.351 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1010 (0x7f431bfde910) [ CSSD]2013-05-23 15:58:13.351 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde910 (RPC#1010) tag(3f2002c) sent to node 1 [ CSSD]2013-05-23 15:58:13.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03f2002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:13.351 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014260, gin 1, member 0 [ CSSD]2013-05-23 15:58:13.351 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:58:13.352 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:58:13.352 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:58:13.352 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde910 (RPC#1010) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:13.352 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde910, ret 0, client 0x7f431c0cf720 member 0x7f431c13ea10 [ CSSD]2013-05-23 15:58:13.352 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 15:58:13.352 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1010 [ CSSD]2013-05-23 15:58:13.358 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 15:58:13.358 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:13.358 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ea10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:13.358 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1011 (0x7f431bfde990) [ CSSD]2013-05-23 15:58:13.358 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde990 (RPC#1011) tag(3f3002c) sent to node 1 [ CSSD]2013-05-23 15:58:13.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f3002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:13.359 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:13.359 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde990 (RPC#1011) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:13.359 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde990, ret 0, client 0x7f431c0cf720 member 0x7f431c13ea10 [ CSSD]2013-05-23 15:58:13.359 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1011 [ CSSD]2013-05-23 15:58:13.359 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13ea10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:58:13.562 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 15:58:13.562 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 26237 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:14.523 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:14.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:14.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:15.551 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:15.551 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:58:16.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:16.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:16.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:17.223 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 15:58:17.224 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:58:17.225 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 15:58:17.225 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c10a190) pid(26270) version 11:1:1:4 [ CSSD]2013-05-23 15:58:17.231 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c10a190), client(1/0x7f431c121510) [ CSSD]2013-05-23 15:58:17.231 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:58:17.231 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:58:17.231 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:17.231 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1012 (0x7f431bfdea10) [ CSSD]2013-05-23 15:58:17.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdea10 (RPC#1012) tag(3f4002c) sent to node 1 [ CSSD]2013-05-23 15:58:17.231 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03f4002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:17.231 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014261, gin 1, member 0 [ CSSD]2013-05-23 15:58:17.231 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:58:17.232 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:58:17.232 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:58:17.232 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdea10 (RPC#1012) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:17.232 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdea10, ret 0, client 0x7f431c121510 member 0x7f431c13f950 [ CSSD]2013-05-23 15:58:17.232 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 15:58:17.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1012 [ CSSD]2013-05-23 15:58:17.238 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 15:58:17.238 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:17.238 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:17.238 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1013 (0x7f431bfdea90) [ CSSD]2013-05-23 15:58:17.238 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdea90 (RPC#1013) tag(3f5002c) sent to node 1 [ CSSD]2013-05-23 15:58:17.238 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f5002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:17.238 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:17.238 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdea90 (RPC#1013) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:17.238 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdea90, ret 0, client 0x7f431c121510 member 0x7f431c13f950 [ CSSD]2013-05-23 15:58:17.238 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1013 [ CSSD]2013-05-23 15:58:17.238 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:58:17.296 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:58:17.296 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1014 (0x7f431bfdeb10) [ CSSD]2013-05-23 15:58:17.296 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdeb10 (RPC#1014) tag(3f6002c) sent to node 1 [ CSSD]2013-05-23 15:58:17.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:17.297 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1014 (0x7f431bfdeb10) [ CSSD]2013-05-23 15:58:17.297 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdeb10 (RPC#1014) state 6, flags 0x202 [ CSSD]2013-05-23 15:58:17.297 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdeb10, ret 0, client (nil) [ CSSD]2013-05-23 15:58:17.297 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1014 [ CSSD]2013-05-23 15:58:17.441 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 15:58:17.441 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0840c0) skgpid ospid 26270 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:18.599 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:18.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:18.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:20.554 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:20.554 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:20.605 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:20.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:20.905 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:22.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:22.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:22.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:24.623 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:24.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:24.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:25.557 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:25.558 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:26.647 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:26.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:26.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:27.323 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:58:27.323 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1015 (0x7f431bfdeb90) [ CSSD]2013-05-23 15:58:27.323 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdeb90 (RPC#1015) tag(3f7002c) sent to node 1 [ CSSD]2013-05-23 15:58:27.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:27.323 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1015 (0x7f431bfdeb90) [ CSSD]2013-05-23 15:58:27.324 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdeb90 (RPC#1015) state 6, flags 0x202 [ CSSD]2013-05-23 15:58:27.324 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdeb90, ret 0, client (nil) [ CSSD]2013-05-23 15:58:27.324 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1015 [ CSSD]2013-05-23 15:58:27.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:27.724 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014262, gin 1, member 0 [ CSSD]2013-05-23 15:58:27.724 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f18e0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:58:27.724 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:58:27.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:27.731 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:58:27.731 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7a00) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:58:28.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:28.961 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:28.961 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:30.560 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:30.560 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:30.707 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:30.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:30.985 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:32.473 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 15:58:32.474 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:58:32.474 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 15:58:32.474 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c11fd30) pid(26411) version 11:1:1:4 [ CSSD]2013-05-23 15:58:32.480 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 15:58:32.480 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:58:32.480 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:58:32.480 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:32.480 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1016 (0x7f431bfdec10) [ CSSD]2013-05-23 15:58:32.480 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdec10 (RPC#1016) tag(3f8002c) sent to node 1 [ CSSD]2013-05-23 15:58:32.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03f8002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:32.480 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014263, gin 1, member 0 [ CSSD]2013-05-23 15:58:32.480 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:58:32.481 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:58:32.481 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:58:32.481 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdec10 (RPC#1016) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:32.481 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdec10, ret 0, client 0x7f431c12e6d0 member 0x7f431c13ea10 [ CSSD]2013-05-23 15:58:32.481 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 15:58:32.481 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1016 [ CSSD]2013-05-23 15:58:32.487 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:58:32.487 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:32.487 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ea10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:32.487 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1017 (0x7f431bfdec90) [ CSSD]2013-05-23 15:58:32.487 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdec90 (RPC#1017) tag(3f9002c) sent to node 1 [ CSSD]2013-05-23 15:58:32.488 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f9002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:32.488 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:32.488 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdec90 (RPC#1017) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:32.488 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdec90, ret 0, client 0x7f431c12e6d0 member 0x7f431c13ea10 [ CSSD]2013-05-23 15:58:32.488 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1017 [ CSSD]2013-05-23 15:58:32.488 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13ea10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:58:32.690 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 15:58:32.690 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12f0c0) skgpid ospid 26411 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:32.719 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:32.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:32.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:34.563 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:34.563 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:58:34.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:35.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:35.043 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:36.749 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:36.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:36.864 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014264, gin 1, member 0 [ CSSD]2013-05-23 15:58:36.864 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f18e0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 15:58:36.864 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:58:36.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:36.871 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 15:58:36.871 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:58:36.996 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:58:36.996 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:58:36.998 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:58:36.998 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:58:36.999 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:58:37.000 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:58:37.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:37.048 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:37.349 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:58:37.350 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1018 (0x7f431bfded10) [ CSSD]2013-05-23 15:58:37.350 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfded10 (RPC#1018) tag(3fa002c) sent to node 1 [ CSSD]2013-05-23 15:58:37.350 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:37.350 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1018 (0x7f431bfded10) [ CSSD]2013-05-23 15:58:37.350 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfded10 (RPC#1018) state 6, flags 0x202 [ CSSD]2013-05-23 15:58:37.350 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfded10, ret 0, client (nil) [ CSSD]2013-05-23 15:58:37.351 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1018 [ CSSD]2013-05-23 15:58:38.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:39.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:39.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:39.566 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:39.567 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:40.805 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:41.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:41.067 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:41.732 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 15:58:41.733 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:58:41.734 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 15:58:41.734 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c10a190) pid(26011/26011) version 11:1:1:4 [ CSSD]2013-05-23 15:58:41.740 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c10a190), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 15:58:41.740 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 15:58:41.740 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:58:41.740 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 15:58:42.713 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c10a190), client(2/0x7f431c12e6d0) [ CSSD]2013-05-23 15:58:42.713 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 15:58:42.713 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 15:58:42.713 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 15:58:42.819 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:42.972 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:42.972 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014265, gin 1, member 0 [ CSSD]2013-05-23 15:58:42.972 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f18e0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:58:42.972 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:58:42.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:42.978 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:58:42.978 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:58:43.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:43.073 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:44.570 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:44.570 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:44.825 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:45.084 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:45.084 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:46.850 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:47.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:47.109 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:47.375 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:58:47.375 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1019 (0x7f431bfded90) [ CSSD]2013-05-23 15:58:47.375 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfded90 (RPC#1019) tag(3fb002c) sent to node 1 [ CSSD]2013-05-23 15:58:47.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:47.375 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1019 (0x7f431bfded90) [ CSSD]2013-05-23 15:58:47.375 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfded90 (RPC#1019) state 6, flags 0x202 [ CSSD]2013-05-23 15:58:47.375 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfded90, ret 0, client (nil) [ CSSD]2013-05-23 15:58:47.376 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1019 [ CSSD]2013-05-23 15:58:47.733 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13edd0) allocated [ CSSD]2013-05-23 15:58:47.734 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:58:47.735 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13edd0 - 1,2,3 [ CSSD]2013-05-23 15:58:47.735 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c13edd0) pid(26552) version 11:1:1:4 [ CSSD]2013-05-23 15:58:47.740 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c13edd0), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:58:47.740 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:58:47.740 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c109f70, requested num -1 [ CSSD]2013-05-23 15:58:47.741 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:47.741 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1020 (0x7f431bfdee10) [ CSSD]2013-05-23 15:58:47.741 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdee10 (RPC#1020) tag(3fc002c) sent to node 1 [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03fc002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014266, gin 1, member 0 [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdee10 (RPC#1020) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdee10, ret 0, client 0x7f431c10a560 member 0x7f431c1086b0 [ CSSD]2013-05-23 15:58:47.741 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:58:47.742 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1020 [ CSSD]2013-05-23 15:58:47.748 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:58:47.748 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:47.748 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1086b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:58:47.748 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1021 (0x7f431bfdee90) [ CSSD]2013-05-23 15:58:47.748 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdee90 (RPC#1021) tag(3fd002c) sent to node 1 [ CSSD]2013-05-23 15:58:47.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03fd002c) incarnation(174631051) [ CSSD]2013-05-23 15:58:47.748 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:58:47.748 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdee90 (RPC#1021) state 6, flags 0x100 [ CSSD]2013-05-23 15:58:47.748 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdee90, ret 0, client 0x7f431c10a560 member 0x7f431c1086b0 [ CSSD]2013-05-23 15:58:47.748 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1021 [ CSSD]2013-05-23 15:58:47.749 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1086b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:58:47.951 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13edd0 [ CSSD]2013-05-23 15:58:47.951 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13edd0) con(0x7f431c0e6e20) skgpid ospid 26552 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:48.901 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:49.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:49.142 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:49.573 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:49.573 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:50.907 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:51.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:51.149 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:52.918 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:53.118 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ee1b0) [ CSSD]2013-05-23 15:58:53.118 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ee1b0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:58:53.118 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ee1b0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:58:53.118 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 15:58:53.118 [1191196992] >TRACE: clssgmDeadProc: proc 0x23178a0 [ CSSD]2013-05-23 15:58:53.118 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23178a0) con(0x22f2150) skgpid 21313 ospid 21313 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:53.128 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22e8180) [ CSSD]2013-05-23 15:58:53.128 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22e8180), grp DBHWPROD, member 1 [ CSSD]2013-05-23 15:58:53.128 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22e8180), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 15:58:53.128 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 15:58:53.137 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f2350) [ CSSD]2013-05-23 15:58:53.137 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f2350), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 15:58:53.137 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f2350), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 15:58:53.137 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 15:58:53.137 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 15:58:53.137 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c0e6720) skgpid 21317 ospid 21317 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:58:53.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:53.174 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:53.576 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:53.576 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:58:54.925 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:55.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:55.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:56.951 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:57.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:57.192 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:58:57.457 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:58:57.458 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1022 (0x7f431bfdef10) [ CSSD]2013-05-23 15:58:57.458 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdef10 (RPC#1022) tag(3fe002c) sent to node 1 [ CSSD]2013-05-23 15:58:57.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:57.458 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1022 (0x7f431bfdef10) [ CSSD]2013-05-23 15:58:57.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdef10 (RPC#1022) state 6, flags 0x202 [ CSSD]2013-05-23 15:58:57.458 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdef10, ret 0, client (nil) [ CSSD]2013-05-23 15:58:57.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1022 [ CSSD]2013-05-23 15:58:58.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:58.232 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014267, gin 1, member 0 [ CSSD]2013-05-23 15:58:58.232 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:58:58.232 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:58:58.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:58.239 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:58:58.239 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:58:58.579 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:58:58.579 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:58:58.982 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:58:59.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:58:59.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:01.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:01.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:01.249 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:02.982 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 15:59:02.983 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:59:02.984 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 15:59:02.984 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c129c20) pid(26712) version 11:1:1:4 [ CSSD]2013-05-23 15:59:02.990 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129c20), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:59:02.990 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:02.990 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 15:59:02.990 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:02.990 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1023 (0x7f431bfdef90) [ CSSD]2013-05-23 15:59:02.990 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdef90 (RPC#1023) tag(3ff002c) sent to node 1 [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ff002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014268, gin 1, member 0 [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdef90 (RPC#1023) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdef90, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:59:02.991 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1023 [ CSSD]2013-05-23 15:59:02.997 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:02.997 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:02.997 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:02.997 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1024 (0x7f431bfdf010) [ CSSD]2013-05-23 15:59:02.998 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf010 (RPC#1024) tag(400002c) sent to node 1 [ CSSD]2013-05-23 15:59:02.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0400002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:02.998 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:02.998 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf010 (RPC#1024) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:02.998 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf010, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:02.998 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1024 [ CSSD]2013-05-23 15:59:02.998 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:59:03.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:03.201 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 15:59:03.201 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c11ab50) skgpid ospid 26712 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:59:03.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:03.255 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:03.582 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:03.583 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:05.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:05.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:05.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:07.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:07.290 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:07.290 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:07.481 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:59:07.481 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1025 (0x7f431bfdf090) [ CSSD]2013-05-23 15:59:07.481 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf090 (RPC#1025) tag(401002c) sent to node 1 [ CSSD]2013-05-23 15:59:07.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:07.482 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1025 (0x7f431bfdf090) [ CSSD]2013-05-23 15:59:07.482 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf090 (RPC#1025) state 6, flags 0x202 [ CSSD]2013-05-23 15:59:07.482 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf090, ret 0, client (nil) [ CSSD]2013-05-23 15:59:07.482 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1025 [ CSSD]2013-05-23 15:59:08.585 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:08.586 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:09.122 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:09.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:09.343 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:09.832 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:09.832 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014269, gin 1, member 0 [ CSSD]2013-05-23 15:59:09.832 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:59:09.832 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:59:09.839 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:09.839 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:59:09.839 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:59:11.128 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:11.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:11.368 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:13.150 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:13.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:13.373 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:13.483 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:13.483 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014270, gin 1, member 0 [ CSSD]2013-05-23 15:59:13.483 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:59:13.483 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:59:13.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:13.490 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:59:13.490 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:59:13.589 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:13.589 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:13.603 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 15:59:13.604 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:59:13.605 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 15:59:13.605 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c129c20) pid(26824) version 11:1:1:4 [ CSSD]2013-05-23 15:59:13.611 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129c20), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:59:13.611 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:13.611 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 15:59:13.611 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:13.611 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1026 (0x7f431bfdf110) [ CSSD]2013-05-23 15:59:13.611 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf110 (RPC#1026) tag(402002c) sent to node 1 [ CSSD]2013-05-23 15:59:13.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0402002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:13.611 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014271, gin 1, member 0 [ CSSD]2013-05-23 15:59:13.611 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:59:13.612 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:59:13.612 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:59:13.612 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf110 (RPC#1026) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:13.612 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf110, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:13.612 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:59:13.612 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1026 [ CSSD]2013-05-23 15:59:13.618 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:13.618 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:13.618 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:13.618 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1027 (0x7f431bfdf190) [ CSSD]2013-05-23 15:59:13.618 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf190 (RPC#1027) tag(403002c) sent to node 1 [ CSSD]2013-05-23 15:59:13.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0403002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:13.619 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:13.619 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf190 (RPC#1027) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:13.619 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf190, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:13.619 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1027 [ CSSD]2013-05-23 15:59:13.619 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:59:13.821 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 15:59:13.821 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c11ab50) skgpid ospid 26824 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:59:15.156 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:15.386 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:15.386 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:17.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:17.392 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:17.392 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:17.507 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:59:17.507 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1028 (0x7f431bfdf210) [ CSSD]2013-05-23 15:59:17.508 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf210 (RPC#1028) tag(404002c) sent to node 1 [ CSSD]2013-05-23 15:59:17.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:17.508 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1028 (0x7f431bfdf210) [ CSSD]2013-05-23 15:59:17.508 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf210 (RPC#1028) state 6, flags 0x202 [ CSSD]2013-05-23 15:59:17.508 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf210, ret 0, client (nil) [ CSSD]2013-05-23 15:59:17.508 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1028 [ CSSD]2013-05-23 15:59:18.233 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 15:59:18.234 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:59:18.234 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 15:59:18.234 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c129c20) pid(26859) version 11:1:1:4 [ CSSD]2013-05-23 15:59:18.240 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129c20), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:59:18.240 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:18.240 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c135110, requested num -1 [ CSSD]2013-05-23 15:59:18.240 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:18.240 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1029 (0x7f431bfdf290) [ CSSD]2013-05-23 15:59:18.241 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf290 (RPC#1029) tag(405002c) sent to node 1 [ CSSD]2013-05-23 15:59:18.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0405002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:18.241 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014272, gin 1, member 0 [ CSSD]2013-05-23 15:59:18.241 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:59:18.242 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:59:18.242 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:59:18.242 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf290 (RPC#1029) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:18.242 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf290, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:18.242 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:59:18.242 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1029 [ CSSD]2013-05-23 15:59:18.248 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:18.248 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:18.248 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:18.248 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1030 (0x7f431bfdf310) [ CSSD]2013-05-23 15:59:18.248 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf310 (RPC#1030) tag(406002c) sent to node 1 [ CSSD]2013-05-23 15:59:18.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0406002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:18.249 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:18.249 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf310 (RPC#1030) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:18.249 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf310, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:18.249 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1030 [ CSSD]2013-05-23 15:59:18.249 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:59:18.451 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 15:59:18.451 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0e6e20) skgpid ospid 26859 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:59:18.592 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:18.592 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:19.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:19.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:19.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:21.218 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:21.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:21.449 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:23.231 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:23.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:23.455 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:23.594 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:23.595 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:23.838 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:59:23.838 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:59:23.839 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:59:23.839 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 15:59:23.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:23.921 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014273, gin 1, member 0 [ CSSD]2013-05-23 15:59:23.921 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f1f70/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 15:59:23.921 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 15:59:23.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:23.927 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 15:59:23.927 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:59:25.235 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:25.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:25.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:27.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:27.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:27.473 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:27.533 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:59:27.534 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1031 (0x7f431bfdf390) [ CSSD]2013-05-23 15:59:27.534 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf390 (RPC#1031) tag(407002c) sent to node 1 [ CSSD]2013-05-23 15:59:27.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:27.535 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1031 (0x7f431bfdf390) [ CSSD]2013-05-23 15:59:27.535 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf390 (RPC#1031) state 6, flags 0x202 [ CSSD]2013-05-23 15:59:27.535 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf390, ret 0, client (nil) [ CSSD]2013-05-23 15:59:27.535 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1031 [ CSSD]2013-05-23 15:59:28.598 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:28.598 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:28.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:28.736 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014274, gin 1, member 0 [ CSSD]2013-05-23 15:59:28.736 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:59:28.736 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:59:28.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:28.743 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:59:28.743 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:59:29.294 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:29.483 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:29.483 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:31.300 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:31.488 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:31.488 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:33.311 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:33.484 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 15:59:33.485 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:59:33.485 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 15:59:33.485 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c129c20) pid(27018) version 11:1:1:4 [ CSSD]2013-05-23 15:59:33.493 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:33.493 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:33.502 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129c20), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:59:33.502 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:33.502 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:59:33.502 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:33.502 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1032 (0x7f431bfdf410) [ CSSD]2013-05-23 15:59:33.502 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf410 (RPC#1032) tag(408002c) sent to node 1 [ CSSD]2013-05-23 15:59:33.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0408002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:33.502 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014275, gin 1, member 0 [ CSSD]2013-05-23 15:59:33.502 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:59:33.503 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:59:33.503 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:59:33.503 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf410 (RPC#1032) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:33.503 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf410, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:33.503 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 15:59:33.503 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1032 [ CSSD]2013-05-23 15:59:33.510 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:33.510 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:33.510 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:33.510 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1033 (0x7f431bfdf490) [ CSSD]2013-05-23 15:59:33.510 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf490 (RPC#1033) tag(409002c) sent to node 1 [ CSSD]2013-05-23 15:59:33.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0409002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:33.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:33.511 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf490 (RPC#1033) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:33.511 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf490, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 15:59:33.511 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1033 [ CSSD]2013-05-23 15:59:33.511 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:59:33.601 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:33.601 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:33.713 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 15:59:33.713 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c12f0c0) skgpid ospid 27018 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:59:35.336 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:35.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:35.516 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:37.001 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 15:59:37.002 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 15:59:37.003 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 15:59:37.003 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c129c20) pid(26646/26646) version 11:1:1:4 [ CSSD]2013-05-23 15:59:37.019 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c129c20), client(1/0x7f431c10a560) [ CSSD]2013-05-23 15:59:37.019 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 15:59:37.019 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 15:59:37.019 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 15:59:37.389 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:37.521 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:37.521 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:37.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:59:37.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1034 (0x7f431bfdf510) [ CSSD]2013-05-23 15:59:37.556 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf510 (RPC#1034) tag(40a002c) sent to node 1 [ CSSD]2013-05-23 15:59:37.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:37.557 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1034 (0x7f431bfdf510) [ CSSD]2013-05-23 15:59:37.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf510 (RPC#1034) state 6, flags 0x202 [ CSSD]2013-05-23 15:59:37.557 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf510, ret 0, client (nil) [ CSSD]2013-05-23 15:59:37.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1034 [ CSSD]2013-05-23 15:59:37.603 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:37.604 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 15:59:39.395 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:39.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:39.554 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:41.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:41.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:41.560 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:42.606 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:42.607 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:43.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:43.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:43.566 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:43.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:43.986 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014276, gin 1, member 0 [ CSSD]2013-05-23 15:59:43.986 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:59:43.986 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:59:43.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:43.993 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:59:43.993 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:59:45.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:45.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:45.576 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:47.490 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:47.580 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:59:47.580 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1035 (0x7f431bfdf590) [ CSSD]2013-05-23 15:59:47.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf590 (RPC#1035) tag(40b002c) sent to node 1 [ CSSD]2013-05-23 15:59:47.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:47.581 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1035 (0x7f431bfdf590) [ CSSD]2013-05-23 15:59:47.581 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf590 (RPC#1035) state 6, flags 0x202 [ CSSD]2013-05-23 15:59:47.581 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf590, ret 0, client (nil) [ CSSD]2013-05-23 15:59:47.581 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1035 [ CSSD]2013-05-23 15:59:47.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:47.582 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:47.609 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:47.609 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:48.746 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 15:59:48.747 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 15:59:48.747 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 15:59:48.747 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c0958d0) pid(27174) version 11:1:1:4 [ CSSD]2013-05-23 15:59:48.754 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c0958d0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 15:59:48.754 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:59:48.754 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 15:59:48.754 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:48.754 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1036 (0x7f431bfdf610) [ CSSD]2013-05-23 15:59:48.754 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf610 (RPC#1036) tag(40c002c) sent to node 1 [ CSSD]2013-05-23 15:59:48.754 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(040c002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:48.754 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014277, gin 1, member 0 [ CSSD]2013-05-23 15:59:48.754 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 15:59:48.754 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 15:59:48.754 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 15:59:48.755 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf610 (RPC#1036) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:48.755 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf610, ret 0, client 0x7f431c0e5910 member 0x7f431c108a80 [ CSSD]2013-05-23 15:59:48.755 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 15:59:48.755 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1036 [ CSSD]2013-05-23 15:59:48.761 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 15:59:48.761 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:48.761 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 15:59:48.761 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1037 (0x7f431bfdf690) [ CSSD]2013-05-23 15:59:48.762 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf690 (RPC#1037) tag(40d002c) sent to node 1 [ CSSD]2013-05-23 15:59:48.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(040d002c) incarnation(174631051) [ CSSD]2013-05-23 15:59:48.762 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 15:59:48.762 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf690 (RPC#1037) state 6, flags 0x100 [ CSSD]2013-05-23 15:59:48.762 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf690, ret 0, client 0x7f431c0e5910 member 0x7f431c108a80 [ CSSD]2013-05-23 15:59:48.762 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1037 [ CSSD]2013-05-23 15:59:48.762 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 15:59:48.965 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 15:59:48.965 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c0e6720) skgpid ospid 27174 with 0 clients, refcount 0 [ CSSD]2013-05-23 15:59:49.496 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:49.593 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:49.593 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:51.520 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:51.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:51.599 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:52.612 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:52.612 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:53.533 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:53.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:53.652 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:55.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:55.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:55.658 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:57.590 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:57.606 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 15:59:57.606 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1038 (0x7f431bfdf710) [ CSSD]2013-05-23 15:59:57.606 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf710 (RPC#1038) tag(40e002c) sent to node 1 [ CSSD]2013-05-23 15:59:57.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:57.607 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1038 (0x7f431bfdf710) [ CSSD]2013-05-23 15:59:57.607 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf710 (RPC#1038) state 6, flags 0x202 [ CSSD]2013-05-23 15:59:57.607 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf710, ret 0, client (nil) [ CSSD]2013-05-23 15:59:57.607 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1038 [ CSSD]2013-05-23 15:59:57.615 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 15:59:57.615 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 15:59:57.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:57.663 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 15:59:59.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:59.236 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014278, gin 1, member 0 [ CSSD]2013-05-23 15:59:59.236 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 15:59:59.236 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 15:59:59.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:59.242 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 15:59:59.242 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 15:59:59.595 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 15:59:59.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 15:59:59.674 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:01.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:01.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:01.679 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:02.618 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:02.618 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:03.633 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:03.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:03.693 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:03.997 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 16:00:03.997 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:00:03.998 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 16:00:03.998 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c0958d0) pid(27503) version 11:1:1:4 [ CSSD]2013-05-23 16:00:04.005 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c0958d0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:00:04.005 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:00:04.005 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:00:04.005 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:04.005 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1039 (0x7f431bfdf790) [ CSSD]2013-05-23 16:00:04.005 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf790 (RPC#1039) tag(40f002c) sent to node 1 [ CSSD]2013-05-23 16:00:04.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(040f002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:04.005 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014279, gin 1, member 0 [ CSSD]2013-05-23 16:00:04.005 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:00:04.006 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:00:04.006 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:00:04.006 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf790 (RPC#1039) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:04.006 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf790, ret 0, client 0x7f431c0e5910 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:04.006 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:00:04.006 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1039 [ CSSD]2013-05-23 16:00:04.013 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:00:04.013 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:04.013 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:04.013 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1040 (0x7f431bfdf810) [ CSSD]2013-05-23 16:00:04.013 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf810 (RPC#1040) tag(410002c) sent to node 1 [ CSSD]2013-05-23 16:00:04.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0410002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:04.013 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:04.013 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf810 (RPC#1040) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:04.013 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf810, ret 0, client 0x7f431c0e5910 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:04.013 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1040 [ CSSD]2013-05-23 16:00:04.014 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:00:04.216 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 16:00:04.216 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c11ab50) skgpid ospid 27503 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:05.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:05.698 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:05.698 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:07.621 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:07.621 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:07.634 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:00:07.634 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1041 (0x7f431bfdf890) [ CSSD]2013-05-23 16:00:07.634 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf890 (RPC#1041) tag(411002c) sent to node 1 [ CSSD]2013-05-23 16:00:07.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:07.635 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1041 (0x7f431bfdf890) [ CSSD]2013-05-23 16:00:07.635 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf890 (RPC#1041) state 6, flags 0x202 [ CSSD]2013-05-23 16:00:07.635 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf890, ret 0, client (nil) [ CSSD]2013-05-23 16:00:07.635 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1041 [ CSSD]2013-05-23 16:00:07.691 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:07.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:07.703 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:08.059 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 16:00:08.060 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:00:08.060 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 16:00:08.060 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c0958d0) pid(27580) version 11:1:1:4 [ CSSD]2013-05-23 16:00:08.663 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 16:00:08.663 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c0e6e20) skgpid ospid 27580 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:09.697 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:09.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:09.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:10.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:10.083 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014280, gin 1, member 0 [ CSSD]2013-05-23 16:00:10.083 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:00:10.083 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:00:10.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:10.089 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:00:10.089 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e7df0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:00:11.702 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:11.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:11.759 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:12.624 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:12.624 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:13.713 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:13.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:13.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:13.857 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 16:00:13.858 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:00:13.859 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 16:00:13.859 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c0958d0) pid(27722) version 11:1:1:4 [ CSSD]2013-05-23 16:00:13.875 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c0958d0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:00:13.875 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:13.875 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:00:13.875 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:13.875 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1042 (0x7f431bfdf910) [ CSSD]2013-05-23 16:00:13.876 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf910 (RPC#1042) tag(412002c) sent to node 1 [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0412002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014281, gin 1, member 0 [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf910 (RPC#1042) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf910, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:13.876 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:00:13.877 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1042 [ CSSD]2013-05-23 16:00:13.884 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:13.884 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:13.884 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:13.884 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1043 (0x7f431bfdf990) [ CSSD]2013-05-23 16:00:13.884 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf990 (RPC#1043) tag(413002c) sent to node 1 [ CSSD]2013-05-23 16:00:13.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0413002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:13.884 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:13.884 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf990 (RPC#1043) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:13.884 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf990, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:13.885 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1043 [ CSSD]2013-05-23 16:00:13.885 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:00:14.087 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 16:00:14.087 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c108e60) skgpid ospid 27722 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:14.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:14.496 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014282, gin 1, member 0 [ CSSD]2013-05-23 16:00:14.496 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f1f70/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:00:14.496 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:00:14.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:14.503 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:00:14.503 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e7df0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:00:15.739 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:15.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:15.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:17.628 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:17.628 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:17.671 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:00:17.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1044 (0x7f431bfdfa10) [ CSSD]2013-05-23 16:00:17.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfa10 (RPC#1044) tag(414002c) sent to node 1 [ CSSD]2013-05-23 16:00:17.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:17.673 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1044 (0x7f431bfdfa10) [ CSSD]2013-05-23 16:00:17.673 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfa10 (RPC#1044) state 6, flags 0x202 [ CSSD]2013-05-23 16:00:17.673 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfa10, ret 0, client (nil) [ CSSD]2013-05-23 16:00:17.673 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1044 [ CSSD]2013-05-23 16:00:17.792 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:17.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:17.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:18.851 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e7b70) [ CSSD]2013-05-23 16:00:18.851 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e7b70), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:00:18.851 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e7b70), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:00:18.851 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:00:18.851 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e75d0 [ CSSD]2013-05-23 16:00:18.851 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e75d0) con(0x2320f80) skgpid 22466 ospid 22466 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:18.852 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22e4ef0) [ CSSD]2013-05-23 16:00:18.852 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22e4ef0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:00:18.852 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22e4ef0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:00:18.852 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 16:00:18.857 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2319520) [ CSSD]2013-05-23 16:00:18.857 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2319520), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:00:18.857 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2319520), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:00:18.857 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:00:18.858 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191f0 [ CSSD]2013-05-23 16:00:18.858 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191f0) con(0x2318360) skgpid 22468 ospid 22468 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:19.257 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 16:00:19.258 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:00:19.258 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 16:00:19.258 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0958d0) pid(27763) version 11:1:1:4 [ CSSD]2013-05-23 16:00:19.263 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0958d0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:00:19.263 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:19.263 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:00:19.263 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:19.263 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1045 (0x7f431bfdfa90) [ CSSD]2013-05-23 16:00:19.264 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfa90 (RPC#1045) tag(415002c) sent to node 1 [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0415002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014283, gin 1, member 0 [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfa90 (RPC#1045) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdfa90, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:19.264 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:00:19.265 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1045 [ CSSD]2013-05-23 16:00:19.271 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:19.271 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:19.271 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:19.271 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1046 (0x7f431bfdfb10) [ CSSD]2013-05-23 16:00:19.271 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfb10 (RPC#1046) tag(416002c) sent to node 1 [ CSSD]2013-05-23 16:00:19.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0416002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:19.271 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:19.271 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfb10 (RPC#1046) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:19.271 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdfb10, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:19.272 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1046 [ CSSD]2013-05-23 16:00:19.272 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:00:19.474 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 16:00:19.474 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c12f0c0) skgpid ospid 27763 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:19.798 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:19.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:19.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:21.803 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:21.861 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:21.861 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:22.631 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:22.631 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:23.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:23.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:23.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:25.839 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:25.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:25.879 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:27.339 [1191196992] >TRACE: clssgmAllocProc: (0x23178a0) allocated [ CSSD]2013-05-23 16:00:27.340 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:00:27.340 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23178a0 - 1,2,3 [ CSSD]2013-05-23 16:00:27.340 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e7ec0) proc(0x23178a0) pid(27857/27857) version 11:1:1:4 [ CSSD]2013-05-23 16:00:27.357 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x23178a0), client(1/0x22f7b50) [ CSSD]2013-05-23 16:00:27.357 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f7b50) [ CSSD]2013-05-23 16:00:27.357 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:00:27.357 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:00:27.432 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x23178a0), client(2/0x22f5110) [ CSSD]2013-05-23 16:00:27.432 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f5110) [ CSSD]2013-05-23 16:00:27.432 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:00:27.432 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 16:00:27.635 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:27.635 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:27.751 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:00:27.752 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1047 (0x7f431bfdfb90) [ CSSD]2013-05-23 16:00:27.752 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfb90 (RPC#1047) tag(417002c) sent to node 1 [ CSSD]2013-05-23 16:00:27.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:27.752 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1047 (0x7f431bfdfb90) [ CSSD]2013-05-23 16:00:27.752 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfb90 (RPC#1047) state 6, flags 0x202 [ CSSD]2013-05-23 16:00:27.752 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfb90, ret 0, client (nil) [ CSSD]2013-05-23 16:00:27.753 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1047 [ CSSD]2013-05-23 16:00:27.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:27.885 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:27.912 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:29.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:29.758 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014284, gin 1, member 0 [ CSSD]2013-05-23 16:00:29.758 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e75d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:00:29.758 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:00:29.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:29.797 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:00:29.797 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e78d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:00:29.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:29.895 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:29.938 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:31.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:31.920 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:31.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:32.638 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:32.638 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:33.945 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:33.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:33.974 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:34.506 [1191196992] >TRACE: clssgmAllocProc: (0x22e3300) allocated [ CSSD]2013-05-23 16:00:34.507 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:00:34.508 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e3300 - 1,2,3 [ CSSD]2013-05-23 16:00:34.508 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e75d0) proc(0x22e3300) pid(27905) version 11:1:1:4 [ CSSD]2013-05-23 16:00:34.513 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e3300), client(1/0x2319520) [ CSSD]2013-05-23 16:00:34.513 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319520) [ CSSD]2013-05-23 16:00:34.513 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319520 with con 0x22e7c30, requested num -1 [ CSSD]2013-05-23 16:00:34.513 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:34.513 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1048 (0x7f431bfdfc10) [ CSSD]2013-05-23 16:00:34.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfc10 (RPC#1048) tag(418002c) sent to node 1 [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0418002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014285, gin 1, member 0 [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3a80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfc10 (RPC#1048) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdfc10, ret 0, client 0x2319520 member 0x22e3d80 [ CSSD]2013-05-23 16:00:34.514 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319520 [ CSSD]2013-05-23 16:00:34.515 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1048 [ CSSD]2013-05-23 16:00:34.520 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319520) [ CSSD]2013-05-23 16:00:34.520 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319520), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:34.520 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e3d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:34.520 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1049 (0x7f431bfdfc90) [ CSSD]2013-05-23 16:00:34.521 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfc90 (RPC#1049) tag(419002c) sent to node 1 [ CSSD]2013-05-23 16:00:34.521 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0419002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:34.521 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:34.521 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfc90 (RPC#1049) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:34.521 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdfc90, ret 0, client 0x2319520 member 0x22e3d80 [ CSSD]2013-05-23 16:00:34.521 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1049 [ CSSD]2013-05-23 16:00:34.521 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e3d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:00:34.724 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e3300 [ CSSD]2013-05-23 16:00:34.724 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e3300) con(0x22e75d0) skgpid ospid 27905 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:35.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:35.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:35.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:36.461 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f5110) [ CSSD]2013-05-23 16:00:36.461 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f5110), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:00:36.461 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f5110), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:00:36.461 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 16:00:36.468 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f7b50) [ CSSD]2013-05-23 16:00:36.468 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f7b50), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:00:36.468 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f7b50), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:00:36.468 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:00:36.468 [1191196992] >TRACE: clssgmDeadProc: proc 0x23178a0 [ CSSD]2013-05-23 16:00:36.468 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23178a0) con(0x22e7ec0) skgpid 27857 ospid 27857 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:36.487 [1191196992] >TRACE: clssgmAllocProc: (0x22f18e0) allocated [ CSSD]2013-05-23 16:00:36.488 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:00:36.488 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f18e0 - 1,2,3 [ CSSD]2013-05-23 16:00:36.488 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e7ec0) proc(0x22f18e0) pid(27960/27960) version 11:1:1:4 [ CSSD]2013-05-23 16:00:36.504 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f18e0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:00:36.504 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:36.504 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:00:36.504 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:00:37.641 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:37.641 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:37.772 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:00:37.772 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1050 (0x7f431bfdfd10) [ CSSD]2013-05-23 16:00:37.772 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfd10 (RPC#1050) tag(41a002c) sent to node 1 [ CSSD]2013-05-23 16:00:37.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:37.773 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1050 (0x7f431bfdfd10) [ CSSD]2013-05-23 16:00:37.773 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfd10 (RPC#1050) state 6, flags 0x202 [ CSSD]2013-05-23 16:00:37.773 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfd10, ret 0, client (nil) [ CSSD]2013-05-23 16:00:37.773 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1050 [ CSSD]2013-05-23 16:00:37.972 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:37.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:37.984 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:38.328 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:38.328 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:00:38.328 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:00:38.328 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:00:38.328 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f18e0 [ CSSD]2013-05-23 16:00:38.328 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f18e0) con(0x22e7ec0) skgpid 27960 ospid 27960 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:39.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:39.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:39.997 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:42.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:42.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:42.003 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:42.644 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:42.644 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:44.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:44.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:44.054 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:45.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:45.020 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014286, gin 1, member 0 [ CSSD]2013-05-23 16:00:45.020 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:00:45.020 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:00:45.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:45.028 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:00:45.028 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:00:46.022 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:46.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:46.059 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:47.647 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:47.647 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:47.803 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:00:47.803 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1051 (0x7f431bfdfd90) [ CSSD]2013-05-23 16:00:47.803 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfd90 (RPC#1051) tag(41b002c) sent to node 1 [ CSSD]2013-05-23 16:00:47.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:47.804 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1051 (0x7f431bfdfd90) [ CSSD]2013-05-23 16:00:47.804 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfd90 (RPC#1051) state 6, flags 0x202 [ CSSD]2013-05-23 16:00:47.804 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfd90, ret 0, client (nil) [ CSSD]2013-05-23 16:00:47.804 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1051 [ CSSD]2013-05-23 16:00:48.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:48.065 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:48.065 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:49.756 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0958d0) allocated [ CSSD]2013-05-23 16:00:49.756 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:00:49.757 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0958d0 - 1,2,3 [ CSSD]2013-05-23 16:00:49.757 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c0958d0) pid(28070) version 11:1:1:4 [ CSSD]2013-05-23 16:00:49.762 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0958d0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:00:49.762 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:49.762 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:00:49.762 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:49.762 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1052 (0x7f431bfdfe10) [ CSSD]2013-05-23 16:00:49.762 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfe10 (RPC#1052) tag(41c002c) sent to node 1 [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(041c002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014287, gin 1, member 0 [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfe10 (RPC#1052) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdfe10, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:00:49.763 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1052 [ CSSD]2013-05-23 16:00:49.770 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:49.770 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:49.770 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:49.770 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1053 (0x7f431bfdfe90) [ CSSD]2013-05-23 16:00:49.770 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfe90 (RPC#1053) tag(41d002c) sent to node 1 [ CSSD]2013-05-23 16:00:49.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(041d002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:49.770 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:49.770 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfe90 (RPC#1053) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:49.770 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdfe90, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:49.770 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1053 [ CSSD]2013-05-23 16:00:49.771 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:00:49.973 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0958d0 [ CSSD]2013-05-23 16:00:49.973 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0958d0) con(0x7f431c11ab50) skgpid ospid 28070 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:50.058 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:50.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:50.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:52.064 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:52.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:52.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:52.651 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:52.651 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:54.116 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:54.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:54.133 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:55.945 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13edd0) allocated [ CSSD]2013-05-23 16:00:55.946 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:00:55.946 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13edd0 - 1,2,3 [ CSSD]2013-05-23 16:00:55.946 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c13edd0) pid(28168) version 11:1:1:4 [ CSSD]2013-05-23 16:00:55.952 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c13edd0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:00:55.952 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:55.952 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:00:55.952 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:55.952 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1054 (0x7f431bfdff10) [ CSSD]2013-05-23 16:00:55.952 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdff10 (RPC#1054) tag(41e002c) sent to node 1 [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(041e002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014288, gin 1, member 0 [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdff10 (RPC#1054) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdff10, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:00:55.953 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1054 [ CSSD]2013-05-23 16:00:55.960 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:00:55.960 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:55.960 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:00:55.960 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1055 (0x7f431bfdff90) [ CSSD]2013-05-23 16:00:55.960 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdff90 (RPC#1055) tag(41f002c) sent to node 1 [ CSSD]2013-05-23 16:00:55.960 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(041f002c) incarnation(174631051) [ CSSD]2013-05-23 16:00:55.960 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:00:55.960 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdff90 (RPC#1055) state 6, flags 0x100 [ CSSD]2013-05-23 16:00:55.960 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdff90, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:00:55.960 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1055 [ CSSD]2013-05-23 16:00:55.960 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:00:56.122 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:56.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:56.139 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:00:56.162 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13edd0 [ CSSD]2013-05-23 16:00:56.162 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13edd0) con(0x7f431c0e6e20) skgpid ospid 28168 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:00:57.654 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:00:57.654 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:00:57.833 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:00:57.833 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1056 (0x7f431bfe0010) [ CSSD]2013-05-23 16:00:57.833 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0010 (RPC#1056) tag(420002c) sent to node 1 [ CSSD]2013-05-23 16:00:57.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:57.834 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1056 (0x7f431bfe0010) [ CSSD]2013-05-23 16:00:57.834 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0010 (RPC#1056) state 6, flags 0x202 [ CSSD]2013-05-23 16:00:57.834 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0010, ret 0, client (nil) [ CSSD]2013-05-23 16:00:57.834 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1056 [ CSSD]2013-05-23 16:00:58.133 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:00:58.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:00:58.145 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:00.138 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:00.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:00.177 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:00.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:00.291 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014289, gin 1, member 0 [ CSSD]2013-05-23 16:01:00.291 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:01:00.291 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:01:00.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:00.299 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:01:00.299 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:01:02.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:02.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:02.211 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:02.657 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:02.657 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:04.213 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:04.213 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:04.237 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:05.004 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 16:01:05.005 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:01:05.005 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 16:01:05.005 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c12b920) pid(28247) version 11:1:1:4 [ CSSD]2013-05-23 16:01:05.011 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:01:05.011 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:05.011 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:01:05.011 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:05.011 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1057 (0x7f431bfe0090) [ CSSD]2013-05-23 16:01:05.011 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0090 (RPC#1057) tag(421002c) sent to node 1 [ CSSD]2013-05-23 16:01:05.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0421002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:05.011 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014290, gin 1, member 0 [ CSSD]2013-05-23 16:01:05.011 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:01:05.011 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:01:05.011 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:01:05.012 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0090 (RPC#1057) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:05.012 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0090, ret 0, client 0x7f431c121510 member 0x7f431c13eec0 [ CSSD]2013-05-23 16:01:05.012 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:01:05.012 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1057 [ CSSD]2013-05-23 16:01:05.018 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:05.018 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:05.018 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eec0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:05.018 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1058 (0x7f431bfe0110) [ CSSD]2013-05-23 16:01:05.018 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0110 (RPC#1058) tag(422002c) sent to node 1 [ CSSD]2013-05-23 16:01:05.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0422002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:05.018 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:05.018 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0110 (RPC#1058) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:05.018 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0110, ret 0, client 0x7f431c121510 member 0x7f431c13eec0 [ CSSD]2013-05-23 16:01:05.018 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1058 [ CSSD]2013-05-23 16:01:05.018 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eec0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:01:05.221 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 16:01:05.221 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c0e6e20) skgpid ospid 28247 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:06.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:06.219 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:06.242 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:07.660 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:07.660 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:07.863 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:01:07.863 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1059 (0x7f431bfe0190) [ CSSD]2013-05-23 16:01:07.863 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0190 (RPC#1059) tag(423002c) sent to node 1 [ CSSD]2013-05-23 16:01:07.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:07.864 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1059 (0x7f431bfe0190) [ CSSD]2013-05-23 16:01:07.864 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0190 (RPC#1059) state 6, flags 0x202 [ CSSD]2013-05-23 16:01:07.864 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0190, ret 0, client (nil) [ CSSD]2013-05-23 16:01:07.864 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1059 [ CSSD]2013-05-23 16:01:08.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:08.225 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:08.254 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:10.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:10.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:10.278 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:10.342 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:10.342 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014291, gin 1, member 0 [ CSSD]2013-05-23 16:01:10.342 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:01:10.342 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:01:10.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:10.349 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:01:10.349 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:01:12.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:12.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:12.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:12.663 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:12.663 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:14.134 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 16:01:14.135 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:01:14.135 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 16:01:14.135 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c12b920) pid(28345) version 11:1:1:4 [ CSSD]2013-05-23 16:01:14.141 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:01:14.141 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:14.141 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:01:14.141 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:14.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1060 (0x7f431bfe0210) [ CSSD]2013-05-23 16:01:14.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0210 (RPC#1060) tag(424002c) sent to node 1 [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0424002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014292, gin 1, member 0 [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0210 (RPC#1060) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0210, ret 0, client 0x7f431c121510 member 0x7f431c13eec0 [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:01:14.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1060 [ CSSD]2013-05-23 16:01:14.149 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:14.149 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:14.149 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eec0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:14.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1061 (0x7f431bfe0290) [ CSSD]2013-05-23 16:01:14.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0290 (RPC#1061) tag(425002c) sent to node 1 [ CSSD]2013-05-23 16:01:14.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0425002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:14.149 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:14.149 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0290 (RPC#1061) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:14.149 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0290, ret 0, client 0x7f431c121510 member 0x7f431c13eec0 [ CSSD]2013-05-23 16:01:14.150 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1061 [ CSSD]2013-05-23 16:01:14.150 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eec0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:01:14.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:14.296 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:14.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:14.352 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 16:01:14.352 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c0e6e20) skgpid ospid 28345 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:15.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:15.556 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014293, gin 1, member 0 [ CSSD]2013-05-23 16:01:15.556 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f80f0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:01:15.556 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:01:15.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:15.595 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:01:15.595 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:01:15.993 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1215d0) allocated [ CSSD]2013-05-23 16:01:15.994 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:01:15.995 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1215d0 - 1,2,3 [ CSSD]2013-05-23 16:01:15.995 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1215d0) pid(28367) version 11:1:1:4 [ CSSD]2013-05-23 16:01:16.001 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1215d0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:01:16.001 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:16.001 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c121510 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:01:16.001 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:01:16.001 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1062 (0x7f431bfe0310) [ CSSD]2013-05-23 16:01:16.002 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0310 (RPC#1062) tag(426002c) sent to node 1 [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0426002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014294, gin 1, member 0 [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0310 (RPC#1062) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0310, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:01:16.002 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1062 [ CSSD]2013-05-23 16:01:16.009 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:16.009 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:01:16.009 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:01:16.009 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1063 (0x7f431bfe0390) [ CSSD]2013-05-23 16:01:16.009 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0390 (RPC#1063) tag(427002c) sent to node 1 [ CSSD]2013-05-23 16:01:16.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0427002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:16.009 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:01:16.009 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0390 (RPC#1063) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:16.009 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0390, ret 0, client 0x7f431c121510 member 0x7f431c108a80 [ CSSD]2013-05-23 16:01:16.009 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1063 [ CSSD]2013-05-23 16:01:16.009 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:01:16.116 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136070) allocated [ CSSD]2013-05-23 16:01:16.117 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:01:16.118 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136070 - 1,2,3 [ CSSD]2013-05-23 16:01:16.118 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c136070) pid(28372) version 11:1:1:4 [ CSSD]2013-05-23 16:01:16.132 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:01:16.133 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:01:16.135 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136070 [ CSSD]2013-05-23 16:01:16.135 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136070) con(0x7f431c108e60) skgpid ospid 28372 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:16.212 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1215d0 [ CSSD]2013-05-23 16:01:16.212 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1215d0) con(0x7f431c12f0c0) skgpid ospid 28367 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:16.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:16.320 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:16.322 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:17.666 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:17.666 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:17.893 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:01:17.893 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1064 (0x7f431bfe0410) [ CSSD]2013-05-23 16:01:17.893 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0410 (RPC#1064) tag(428002c) sent to node 1 [ CSSD]2013-05-23 16:01:17.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:17.894 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1064 (0x7f431bfe0410) [ CSSD]2013-05-23 16:01:17.894 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0410 (RPC#1064) state 6, flags 0x202 [ CSSD]2013-05-23 16:01:17.894 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0410, ret 0, client (nil) [ CSSD]2013-05-23 16:01:17.894 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1064 [ CSSD]2013-05-23 16:01:18.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:18.326 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:18.335 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:20.253 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 16:01:20.254 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:01:20.254 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 16:01:20.254 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c12b920) pid(28409) version 11:1:1:4 [ CSSD]2013-05-23 16:01:20.260 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:01:20.260 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:01:20.260 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:01:20.260 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:20.260 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1065 (0x7f431bfe0490) [ CSSD]2013-05-23 16:01:20.261 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0490 (RPC#1065) tag(429002c) sent to node 1 [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0429002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014295, gin 1, member 0 [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0490 (RPC#1065) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0490, ret 0, client 0x7f431c0e5910 member 0x7f431c1216c0 [ CSSD]2013-05-23 16:01:20.261 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:01:20.262 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1065 [ CSSD]2013-05-23 16:01:20.268 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:01:20.268 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:20.268 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1216c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:20.268 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1066 (0x7f431bfe0510) [ CSSD]2013-05-23 16:01:20.268 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0510 (RPC#1066) tag(42a002c) sent to node 1 [ CSSD]2013-05-23 16:01:20.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(042a002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:20.268 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:20.268 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0510 (RPC#1066) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:20.268 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0510, ret 0, client 0x7f431c0e5910 member 0x7f431c1216c0 [ CSSD]2013-05-23 16:01:20.268 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1066 [ CSSD]2013-05-23 16:01:20.268 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1216c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:01:20.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:20.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:20.359 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:20.471 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 16:01:20.471 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c0e6720) skgpid ospid 28409 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:21.408 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 16:01:21.408 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:01:21.409 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 16:01:21.409 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c12b920) pid(28464/28464) version 11:1:1:4 [ CSSD]2013-05-23 16:01:21.425 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:01:21.425 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:01:21.425 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:01:21.425 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:01:21.439 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:01:21.439 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:01:21.439 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:01:21.439 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:01:21.439 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 16:01:21.439 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c0e6e20) skgpid 28464 ospid 28464 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:21.668 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:21.669 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:01:22.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:22.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:22.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:23.172 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:01:23.172 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2312950), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:01:23.172 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2312950), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:01:23.172 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:01:23.172 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 16:01:23.172 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x205ecb0) skgpid 23175 ospid 23175 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:23.203 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 16:01:23.204 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:01:23.205 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 16:01:23.205 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c12b920) pid(28479/28479) version 11:1:1:4 [ CSSD]2013-05-23 16:01:23.210 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c12b920), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:01:23.210 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:01:23.210 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:01:23.210 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:01:24.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:24.375 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:24.418 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:26.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:26.381 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:26.424 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:26.672 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:26.672 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:27.914 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:01:27.914 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1067 (0x7f431bfe0590) [ CSSD]2013-05-23 16:01:27.914 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0590 (RPC#1067) tag(42b002c) sent to node 1 [ CSSD]2013-05-23 16:01:27.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:27.915 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1067 (0x7f431bfe0590) [ CSSD]2013-05-23 16:01:27.915 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0590 (RPC#1067) state 6, flags 0x202 [ CSSD]2013-05-23 16:01:27.915 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0590, ret 0, client (nil) [ CSSD]2013-05-23 16:01:27.915 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1067 [ CSSD]2013-05-23 16:01:28.387 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:28.387 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:28.493 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:30.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:30.398 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:30.500 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:30.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:30.819 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014296, gin 1, member 0 [ CSSD]2013-05-23 16:01:30.819 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:01:30.819 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:01:30.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:30.825 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:01:30.825 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:01:31.675 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:31.675 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:32.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:32.404 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:32.513 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:34.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:34.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:34.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:35.502 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135f60) allocated [ CSSD]2013-05-23 16:01:35.503 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:01:35.504 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135f60 - 1,2,3 [ CSSD]2013-05-23 16:01:35.504 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c135f60) pid(28590) version 11:1:1:4 [ CSSD]2013-05-23 16:01:35.509 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135f60), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:01:35.509 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:35.509 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:01:35.509 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:35.509 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1068 (0x7f431bfe0610) [ CSSD]2013-05-23 16:01:35.510 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0610 (RPC#1068) tag(42c002c) sent to node 1 [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(042c002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014297, gin 1, member 0 [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0610 (RPC#1068) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0610, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:01:35.510 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1068 [ CSSD]2013-05-23 16:01:35.517 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:35.517 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:35.517 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094db0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:35.517 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1069 (0x7f431bfe0690) [ CSSD]2013-05-23 16:01:35.517 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0690 (RPC#1069) tag(42d002c) sent to node 1 [ CSSD]2013-05-23 16:01:35.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(042d002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:35.517 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:35.517 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0690 (RPC#1069) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:35.517 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0690, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:01:35.517 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1069 [ CSSD]2013-05-23 16:01:35.517 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094db0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:01:35.720 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135f60 [ CSSD]2013-05-23 16:01:35.720 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135f60) con(0x7f431c0e6e20) skgpid ospid 28590 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:36.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:36.421 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:36.543 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:36.678 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:36.679 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:37.934 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:01:37.935 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1070 (0x7f431bfe0710) [ CSSD]2013-05-23 16:01:37.935 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0710 (RPC#1070) tag(42e002c) sent to node 1 [ CSSD]2013-05-23 16:01:37.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:37.935 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1070 (0x7f431bfe0710) [ CSSD]2013-05-23 16:01:37.935 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0710 (RPC#1070) state 6, flags 0x202 [ CSSD]2013-05-23 16:01:37.935 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0710, ret 0, client (nil) [ CSSD]2013-05-23 16:01:37.935 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1070 [ CSSD]2013-05-23 16:01:38.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:38.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:38.555 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:40.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:40.479 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:40.561 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:40.681 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:40.681 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:01:42.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:42.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:42.613 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:44.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:44.514 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:44.619 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:45.684 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:45.684 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:46.084 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:46.084 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014298, gin 1, member 0 [ CSSD]2013-05-23 16:01:46.084 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:01:46.084 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:01:46.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:46.089 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:01:46.089 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:01:46.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:46.520 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:46.624 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:47.954 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:01:47.954 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1071 (0x7f431bfe0790) [ CSSD]2013-05-23 16:01:47.954 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0790 (RPC#1071) tag(42f002c) sent to node 1 [ CSSD]2013-05-23 16:01:47.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:47.955 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1071 (0x7f431bfe0790) [ CSSD]2013-05-23 16:01:47.955 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0790 (RPC#1071) state 6, flags 0x202 [ CSSD]2013-05-23 16:01:47.955 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0790, ret 0, client (nil) [ CSSD]2013-05-23 16:01:47.955 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1071 [ CSSD]2013-05-23 16:01:48.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:48.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:48.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:50.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:50.578 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:50.683 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:50.687 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:50.687 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:50.752 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135f60) allocated [ CSSD]2013-05-23 16:01:50.753 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:01:50.753 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135f60 - 1,2,3 [ CSSD]2013-05-23 16:01:50.753 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c135f60) pid(28739) version 11:1:1:4 [ CSSD]2013-05-23 16:01:50.770 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135f60), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:01:50.770 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:50.770 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:01:50.770 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:50.770 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1072 (0x7f431bfe0810) [ CSSD]2013-05-23 16:01:50.770 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0810 (RPC#1072) tag(430002c) sent to node 1 [ CSSD]2013-05-23 16:01:50.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0430002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:50.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014299, gin 1, member 0 [ CSSD]2013-05-23 16:01:50.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:01:50.771 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:01:50.771 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:01:50.771 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0810 (RPC#1072) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:50.771 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0810, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:01:50.771 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:01:50.771 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1072 [ CSSD]2013-05-23 16:01:50.778 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:01:50.778 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:50.778 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094db0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:01:50.778 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1073 (0x7f431bfe0890) [ CSSD]2013-05-23 16:01:50.778 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0890 (RPC#1073) tag(431002c) sent to node 1 [ CSSD]2013-05-23 16:01:50.778 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0431002c) incarnation(174631051) [ CSSD]2013-05-23 16:01:50.778 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:01:50.778 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0890 (RPC#1073) state 6, flags 0x100 [ CSSD]2013-05-23 16:01:50.778 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0890, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:01:50.778 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1073 [ CSSD]2013-05-23 16:01:50.778 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094db0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:01:50.981 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135f60 [ CSSD]2013-05-23 16:01:50.981 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135f60) con(0x7f431c0e6e20) skgpid ospid 28739 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:01:52.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:52.584 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:52.683 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:54.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:54.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:54.708 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:55.690 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:01:55.690 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:01:56.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:56.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:56.713 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:01:57.974 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:01:57.974 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1074 (0x7f431bfe0910) [ CSSD]2013-05-23 16:01:57.975 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0910 (RPC#1074) tag(432002c) sent to node 1 [ CSSD]2013-05-23 16:01:57.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:57.975 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1074 (0x7f431bfe0910) [ CSSD]2013-05-23 16:01:57.975 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0910 (RPC#1074) state 6, flags 0x202 [ CSSD]2013-05-23 16:01:57.975 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0910, ret 0, client (nil) [ CSSD]2013-05-23 16:01:57.975 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1074 [ CSSD]2013-05-23 16:01:58.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:01:58.608 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:01:58.736 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:00.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:00.617 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:00.693 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:00.693 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:00.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:01.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:01.335 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014300, gin 1, member 0 [ CSSD]2013-05-23 16:02:01.335 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:02:01.336 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:02:01.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:01.343 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:02:01.343 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:02:02.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:02.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:02.753 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:04.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:04.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:04.695 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:04.696 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:02:04.779 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:05.983 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134d40) allocated [ CSSD]2013-05-23 16:02:05.984 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:02:05.985 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134d40 - 1,2,3 [ CSSD]2013-05-23 16:02:05.985 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134d40) pid(28888) version 11:1:1:4 [ CSSD]2013-05-23 16:02:05.993 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134d40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:05.993 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:05.993 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:02:05.993 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:05.993 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1075 (0x7f431bfe0990) [ CSSD]2013-05-23 16:02:05.994 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0990 (RPC#1075) tag(433002c) sent to node 1 [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0433002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014301, gin 1, member 0 [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0990 (RPC#1075) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0990, ret 0, client 0x7f431c121510 member 0x7f431c134c30 [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:05.994 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1075 [ CSSD]2013-05-23 16:02:06.001 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:06.001 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:06.001 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134c30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:06.001 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1076 (0x7f431bfe0a10) [ CSSD]2013-05-23 16:02:06.001 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0a10 (RPC#1076) tag(434002c) sent to node 1 [ CSSD]2013-05-23 16:02:06.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0434002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:06.001 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:06.001 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0a10 (RPC#1076) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:06.001 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0a10, ret 0, client 0x7f431c121510 member 0x7f431c134c30 [ CSSD]2013-05-23 16:02:06.002 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1076 [ CSSD]2013-05-23 16:02:06.002 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134c30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:06.014 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134700) allocated [ CSSD]2013-05-23 16:02:06.015 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:02:06.015 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134700 - 1,2,3 [ CSSD]2013-05-23 16:02:06.015 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c134700) pid(28893) version 11:1:1:4 [ CSSD]2013-05-23 16:02:06.021 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134700), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:06.021 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:06.021 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:02:06.021 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:06.021 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1077 (0x7f431bfe0a90) [ CSSD]2013-05-23 16:02:06.021 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0a90 (RPC#1077) tag(435002c) sent to node 1 [ CSSD]2013-05-23 16:02:06.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0435002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:06.021 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014302, gin 1, member 0 [ CSSD]2013-05-23 16:02:06.021 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091360/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:02:06.022 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:06.022 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:02:06.022 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0a90 (RPC#1077) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:06.022 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0a90, ret 0, client 0x7f431c121510 member 0x7f431c1345f0 [ CSSD]2013-05-23 16:02:06.022 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:06.022 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1077 [ CSSD]2013-05-23 16:02:06.027 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:06.027 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:06.027 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1345f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:06.027 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1078 (0x7f431bfe0b10) [ CSSD]2013-05-23 16:02:06.027 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0b10 (RPC#1078) tag(436002c) sent to node 1 [ CSSD]2013-05-23 16:02:06.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0436002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:06.028 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:06.028 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0b10 (RPC#1078) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:06.028 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0b10, ret 0, client 0x7f431c121510 member 0x7f431c1345f0 [ CSSD]2013-05-23 16:02:06.028 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1078 [ CSSD]2013-05-23 16:02:06.028 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1345f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:06.204 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134d40 [ CSSD]2013-05-23 16:02:06.204 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134d40) con(0x7f431c108e60) skgpid ospid 28888 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:06.230 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134700 [ CSSD]2013-05-23 16:02:06.230 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134700) con(0x7f431c12f0c0) skgpid ospid 28893 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:06.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:06.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:06.783 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:08.004 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:02:08.004 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1079 (0x7f431bfe0b90) [ CSSD]2013-05-23 16:02:08.004 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0b90 (RPC#1079) tag(437002c) sent to node 1 [ CSSD]2013-05-23 16:02:08.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:08.005 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1079 (0x7f431bfe0b90) [ CSSD]2013-05-23 16:02:08.005 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0b90 (RPC#1079) state 6, flags 0x202 [ CSSD]2013-05-23 16:02:08.005 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0b90, ret 0, client (nil) [ CSSD]2013-05-23 16:02:08.005 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1079 [ CSSD]2013-05-23 16:02:08.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:08.685 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:08.837 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:09.699 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:09.699 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:10.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:10.598 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014303, gin 1, member 0 [ CSSD]2013-05-23 16:02:10.598 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:02:10.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:02:10.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:10.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:02:10.605 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:02:10.690 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:10.690 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:10.843 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:12.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:12.695 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:12.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:14.393 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135f60) allocated [ CSSD]2013-05-23 16:02:14.394 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:02:14.394 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135f60 - 1,2,3 [ CSSD]2013-05-23 16:02:14.394 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c135f60) pid(29215) version 11:1:1:4 [ CSSD]2013-05-23 16:02:14.400 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135f60), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:14.400 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:14.400 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:02:14.400 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:14.400 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1080 (0x7f431bfe0c10) [ CSSD]2013-05-23 16:02:14.401 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0c10 (RPC#1080) tag(438002c) sent to node 1 [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0438002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014304, gin 1, member 0 [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0c10 (RPC#1080) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0c10, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:14.401 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1080 [ CSSD]2013-05-23 16:02:14.402 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1346d0) allocated [ CSSD]2013-05-23 16:02:14.403 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:02:14.403 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1346d0 - 1,2,3 [ CSSD]2013-05-23 16:02:14.403 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134d20) proc(0x7f431c1346d0) pid(29218) version 11:1:1:4 [ CSSD]2013-05-23 16:02:14.408 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:14.408 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:14.408 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094db0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:14.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1081 (0x7f431bfe0c90) [ CSSD]2013-05-23 16:02:14.408 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0c90 (RPC#1081) tag(439002c) sent to node 1 [ CSSD]2013-05-23 16:02:14.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0439002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:14.409 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:14.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0c90 (RPC#1081) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:14.409 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0c90, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:02:14.409 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1081 [ CSSD]2013-05-23 16:02:14.409 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094db0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:14.535 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136f00) allocated [ CSSD]2013-05-23 16:02:14.536 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:02:14.536 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136f00 - 1,2,3 [ CSSD]2013-05-23 16:02:14.536 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c136f00) pid(29233) version 11:1:1:4 [ CSSD]2013-05-23 16:02:14.551 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:02:14.552 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:02:14.555 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136f00 [ CSSD]2013-05-23 16:02:14.555 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136f00) con(0x7f431c0e6e20) skgpid ospid 29233 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:14.612 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135f60 [ CSSD]2013-05-23 16:02:14.612 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135f60) con(0x7f431c108e60) skgpid ospid 29215 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:14.631 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(63/0x7f431c121510) [ CSSD]2013-05-23 16:02:14.631 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 63 (0x7f431c121510) [ CSSD]2013-05-23 16:02:14.631 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c121510 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:02:14.631 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:02:14.631 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1082 (0x7f431bfe0d10) [ CSSD]2013-05-23 16:02:14.631 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0d10 (RPC#1082) tag(43a002c) sent to node 1 [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(043a002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014305, gin 1, member 0 [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c1348f0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0d10 (RPC#1082) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0d10, ret 0, client 0x7f431c121510 member 0x7f431c134bf0 [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:14.632 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1082 [ CSSD]2013-05-23 16:02:14.640 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 63 (0x7f431c121510) [ CSSD]2013-05-23 16:02:14.640 [1191196992] >TRACE: clssgmExitGrock: client 63 (0x7f431c121510), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:02:14.640 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134bf0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:02:14.640 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1083 (0x7f431bfe0d90) [ CSSD]2013-05-23 16:02:14.640 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0d90 (RPC#1083) tag(43b002c) sent to node 1 [ CSSD]2013-05-23 16:02:14.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(043b002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:14.640 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:02:14.640 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0d90 (RPC#1083) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:14.640 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0d90, ret 0, client 0x7f431c121510 member 0x7f431c134bf0 [ CSSD]2013-05-23 16:02:14.641 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1083 [ CSSD]2013-05-23 16:02:14.641 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c134bf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:14.642 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1346d0 [ CSSD]2013-05-23 16:02:14.642 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1346d0) con(0x7f431c134d20) skgpid ospid 29218 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:14.702 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:14.702 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:14.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:14.707 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:14.860 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:16.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:16.587 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014306, gin 1, member 0 [ CSSD]2013-05-23 16:02:16.587 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:02:16.587 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:02:16.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:16.594 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:02:16.594 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:02:16.713 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:16.713 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:16.885 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:18.035 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:02:18.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1084 (0x7f431bfe0e10) [ CSSD]2013-05-23 16:02:18.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0e10 (RPC#1084) tag(43c002c) sent to node 1 [ CSSD]2013-05-23 16:02:18.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:18.036 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1084 (0x7f431bfe0e10) [ CSSD]2013-05-23 16:02:18.036 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0e10 (RPC#1084) state 6, flags 0x202 [ CSSD]2013-05-23 16:02:18.036 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0e10, ret 0, client (nil) [ CSSD]2013-05-23 16:02:18.036 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1084 [ CSSD]2013-05-23 16:02:18.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:18.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:18.937 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:19.705 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:19.705 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:20.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:20.731 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:20.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:21.263 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135f60) allocated [ CSSD]2013-05-23 16:02:21.264 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:02:21.264 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135f60 - 1,2,3 [ CSSD]2013-05-23 16:02:21.264 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c135f60) pid(29291) version 11:1:1:4 [ CSSD]2013-05-23 16:02:21.270 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c135f60), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:21.270 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:21.270 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:02:21.270 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:21.270 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1085 (0x7f431bfe0e90) [ CSSD]2013-05-23 16:02:21.270 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0e90 (RPC#1085) tag(43d002c) sent to node 1 [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(043d002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014307, gin 1, member 0 [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0e90 (RPC#1085) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0e90, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:21.271 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1085 [ CSSD]2013-05-23 16:02:21.277 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:21.277 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:21.277 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094db0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:21.277 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1086 (0x7f431bfe0f10) [ CSSD]2013-05-23 16:02:21.277 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0f10 (RPC#1086) tag(43e002c) sent to node 1 [ CSSD]2013-05-23 16:02:21.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(043e002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:21.277 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:21.277 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0f10 (RPC#1086) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:21.278 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0f10, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:02:21.278 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1086 [ CSSD]2013-05-23 16:02:21.278 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094db0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:21.480 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135f60 [ CSSD]2013-05-23 16:02:21.480 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135f60) con(0x7f431c108e60) skgpid ospid 29291 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:22.737 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:22.737 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:22.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:23.708 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:23.708 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:02:24.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:24.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:24.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:26.811 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:26.811 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:26.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:28.064 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:02:28.065 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1087 (0x7f431bfe0f90) [ CSSD]2013-05-23 16:02:28.065 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0f90 (RPC#1087) tag(43f002c) sent to node 1 [ CSSD]2013-05-23 16:02:28.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:28.066 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1087 (0x7f431bfe0f90) [ CSSD]2013-05-23 16:02:28.066 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0f90 (RPC#1087) state 6, flags 0x202 [ CSSD]2013-05-23 16:02:28.066 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0f90, ret 0, client (nil) [ CSSD]2013-05-23 16:02:28.066 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1087 [ CSSD]2013-05-23 16:02:28.711 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:28.711 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:28.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:28.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:29.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:30.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:30.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:31.063 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:31.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:31.846 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014308, gin 1, member 0 [ CSSD]2013-05-23 16:02:31.846 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:02:31.846 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:02:31.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:31.853 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:02:31.853 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:02:32.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:32.937 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:33.076 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:33.714 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:33.714 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:34.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:34.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:35.082 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:36.512 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134d40) allocated [ CSSD]2013-05-23 16:02:36.513 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:02:36.514 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134d40 - 1,2,3 [ CSSD]2013-05-23 16:02:36.514 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c134d40) pid(29472) version 11:1:1:4 [ CSSD]2013-05-23 16:02:36.520 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134d40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:36.520 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:36.520 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:02:36.520 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:36.520 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1088 (0x7f431bfe1010) [ CSSD]2013-05-23 16:02:36.520 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1010 (RPC#1088) tag(440002c) sent to node 1 [ CSSD]2013-05-23 16:02:36.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0440002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:36.520 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014309, gin 1, member 0 [ CSSD]2013-05-23 16:02:36.520 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:02:36.521 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:36.521 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:02:36.521 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1010 (RPC#1088) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:36.521 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1010, ret 0, client 0x7f431c121510 member 0x7f431c134c30 [ CSSD]2013-05-23 16:02:36.521 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:36.521 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1088 [ CSSD]2013-05-23 16:02:36.527 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:36.527 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:36.527 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134c30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:36.527 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1089 (0x7f431bfe1090) [ CSSD]2013-05-23 16:02:36.527 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1090 (RPC#1089) tag(441002c) sent to node 1 [ CSSD]2013-05-23 16:02:36.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0441002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:36.527 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:36.527 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1090 (RPC#1089) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:36.527 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1090, ret 0, client 0x7f431c121510 member 0x7f431c134c30 [ CSSD]2013-05-23 16:02:36.528 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1089 [ CSSD]2013-05-23 16:02:36.528 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134c30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:36.730 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134d40 [ CSSD]2013-05-23 16:02:36.730 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134d40) con(0x7f431c12f0c0) skgpid ospid 29472 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:36.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:36.995 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:37.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:38.085 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:02:38.086 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1090 (0x7f431bfe1110) [ CSSD]2013-05-23 16:02:38.086 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1110 (RPC#1090) tag(442002c) sent to node 1 [ CSSD]2013-05-23 16:02:38.086 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:38.086 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1090 (0x7f431bfe1110) [ CSSD]2013-05-23 16:02:38.086 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1110 (RPC#1090) state 6, flags 0x202 [ CSSD]2013-05-23 16:02:38.086 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1110, ret 0, client (nil) [ CSSD]2013-05-23 16:02:38.086 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1090 [ CSSD]2013-05-23 16:02:38.717 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:38.717 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:39.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:39.006 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:39.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:41.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:41.012 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:41.163 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:42.720 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:42.720 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:02:43.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:43.038 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:43.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:45.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:45.129 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:45.182 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:47.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:47.090 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014310, gin 1, member 0 [ CSSD]2013-05-23 16:02:47.091 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:02:47.091 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:02:47.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:47.097 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:02:47.097 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:02:47.134 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:47.134 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:47.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:47.723 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:47.723 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:48.104 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:02:48.105 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1091 (0x7f431bfe1190) [ CSSD]2013-05-23 16:02:48.105 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1190 (RPC#1091) tag(443002c) sent to node 1 [ CSSD]2013-05-23 16:02:48.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:48.105 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1091 (0x7f431bfe1190) [ CSSD]2013-05-23 16:02:48.105 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1190 (RPC#1091) state 6, flags 0x202 [ CSSD]2013-05-23 16:02:48.105 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1190, ret 0, client (nil) [ CSSD]2013-05-23 16:02:48.106 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1091 [ CSSD]2013-05-23 16:02:48.899 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x23026c0) [ CSSD]2013-05-23 16:02:48.899 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x23026c0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:02:48.899 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x23026c0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:02:48.899 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 16:02:48.900 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2302d90) [ CSSD]2013-05-23 16:02:48.900 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2302d90), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:02:48.900 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2302d90), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:02:48.900 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:02:48.900 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ff670 [ CSSD]2013-05-23 16:02:48.900 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ff670) con(0x2303710) skgpid 24175 ospid 24175 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:48.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x205ed60) [ CSSD]2013-05-23 16:02:48.910 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x205ed60), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:02:48.910 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x205ed60), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:02:48.910 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 16:02:48.910 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 16:02:48.910 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x22d7420) skgpid 24177 ospid 24177 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:49.187 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:49.187 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:49.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:51.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:51.192 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:51.245 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:51.762 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135f60) allocated [ CSSD]2013-05-23 16:02:51.763 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:02:51.763 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135f60 - 1,2,3 [ CSSD]2013-05-23 16:02:51.763 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c135f60) pid(29683) version 11:1:1:4 [ CSSD]2013-05-23 16:02:51.769 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c135f60), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:51.769 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:51.769 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:02:51.769 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:51.769 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1092 (0x7f431bfe1210) [ CSSD]2013-05-23 16:02:51.769 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1210 (RPC#1092) tag(444002c) sent to node 1 [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0444002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014311, gin 1, member 0 [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1210 (RPC#1092) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1210, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:02:51.770 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1092 [ CSSD]2013-05-23 16:02:51.776 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:51.776 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:51.776 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094db0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:02:51.776 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1093 (0x7f431bfe1290) [ CSSD]2013-05-23 16:02:51.776 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1290 (RPC#1093) tag(445002c) sent to node 1 [ CSSD]2013-05-23 16:02:51.777 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0445002c) incarnation(174631051) [ CSSD]2013-05-23 16:02:51.777 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:02:51.777 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1290 (RPC#1093) state 6, flags 0x100 [ CSSD]2013-05-23 16:02:51.777 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1290, ret 0, client 0x7f431c121510 member 0x7f431c094db0 [ CSSD]2013-05-23 16:02:51.777 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1093 [ CSSD]2013-05-23 16:02:51.777 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094db0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:02:51.980 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135f60 [ CSSD]2013-05-23 16:02:51.980 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135f60) con(0x7f431c12f0c0) skgpid ospid 29683 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:02:52.726 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:52.726 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:53.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:53.198 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:53.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:55.006 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135f60) allocated [ CSSD]2013-05-23 16:02:55.007 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:02:55.008 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135f60 - 1,2,3 [ CSSD]2013-05-23 16:02:55.008 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c135f60) pid(28493/28493) version 11:1:1:4 [ CSSD]2013-05-23 16:02:55.013 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c135f60), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:02:55.013 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:02:55.013 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:02:55.013 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 16:02:55.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:55.209 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:55.261 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:57.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:57.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:57.334 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:02:57.729 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:02:57.729 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:02:58.135 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:02:58.136 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1094 (0x7f431bfe1310) [ CSSD]2013-05-23 16:02:58.136 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1310 (RPC#1094) tag(446002c) sent to node 1 [ CSSD]2013-05-23 16:02:58.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:58.136 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1094 (0x7f431bfe1310) [ CSSD]2013-05-23 16:02:58.136 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1310 (RPC#1094) state 6, flags 0x202 [ CSSD]2013-05-23 16:02:58.136 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1310, ret 0, client (nil) [ CSSD]2013-05-23 16:02:58.136 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1094 [ CSSD]2013-05-23 16:02:59.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:02:59.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:02:59.359 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:01.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:01.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:01.363 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:02.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:02.349 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014312, gin 1, member 0 [ CSSD]2013-05-23 16:03:02.349 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:03:02.349 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:03:02.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:02.356 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:03:02.356 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:03:02.731 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:02.732 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:03.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:03.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:03.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:05.290 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:05.290 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:05.400 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:06.733 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:06.734 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:03:07.022 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090fb0) allocated [ CSSD]2013-05-23 16:03:07.023 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:03:07.024 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090fb0 - 1,2,3 [ CSSD]2013-05-23 16:03:07.024 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0952a0) proc(0x7f431c090fb0) pid(29851) version 11:1:1:4 [ CSSD]2013-05-23 16:03:07.029 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c090fb0), client(1/0x7f431c091730) [ CSSD]2013-05-23 16:03:07.029 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091730) [ CSSD]2013-05-23 16:03:07.029 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091730 with con 0x7f431c091240, requested num -1 [ CSSD]2013-05-23 16:03:07.029 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:07.029 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1095 (0x7f431bfe1390) [ CSSD]2013-05-23 16:03:07.029 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1390 (RPC#1095) tag(447002c) sent to node 1 [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0447002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014313, gin 1, member 0 [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0918a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1390 (RPC#1095) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1390, ret 0, client 0x7f431c091730 member 0x7f431c090eb0 [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091730 [ CSSD]2013-05-23 16:03:07.030 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1095 [ CSSD]2013-05-23 16:03:07.036 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091730) [ CSSD]2013-05-23 16:03:07.036 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:07.036 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:07.036 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1096 (0x7f431bfe1410) [ CSSD]2013-05-23 16:03:07.036 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1410 (RPC#1096) tag(448002c) sent to node 1 [ CSSD]2013-05-23 16:03:07.037 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0448002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:07.037 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:07.037 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1410 (RPC#1096) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:07.037 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1410, ret 0, client 0x7f431c091730 member 0x7f431c090eb0 [ CSSD]2013-05-23 16:03:07.037 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1096 [ CSSD]2013-05-23 16:03:07.037 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:03:07.240 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090fb0 [ CSSD]2013-05-23 16:03:07.240 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090fb0) con(0x7f431c0952a0) skgpid ospid 29851 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:03:07.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:07.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:07.435 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:08.155 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:03:08.155 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1097 (0x7f431bfe1490) [ CSSD]2013-05-23 16:03:08.156 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1490 (RPC#1097) tag(449002c) sent to node 1 [ CSSD]2013-05-23 16:03:08.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:08.156 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1097 (0x7f431bfe1490) [ CSSD]2013-05-23 16:03:08.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1490 (RPC#1097) state 6, flags 0x202 [ CSSD]2013-05-23 16:03:08.156 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1490, ret 0, client (nil) [ CSSD]2013-05-23 16:03:08.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1097 [ CSSD]2013-05-23 16:03:09.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:09.347 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:09.388 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090ee0) allocated [ CSSD]2013-05-23 16:03:09.389 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:03:09.390 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090ee0 - 1,2,3 [ CSSD]2013-05-23 16:03:09.390 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c090ee0) pid(29669/29669) version 11:1:1:4 [ CSSD]2013-05-23 16:03:09.395 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c090ee0), client(1/0x7f431c091450) [ CSSD]2013-05-23 16:03:09.395 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c091450) [ CSSD]2013-05-23 16:03:09.395 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:03:09.395 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:03:09.439 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:10.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:10.855 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014314, gin 1, member 0 [ CSSD]2013-05-23 16:03:10.855 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:03:10.855 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:03:10.861 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:10.861 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:03:10.861 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:03:11.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:11.371 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:11.465 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:11.737 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:11.737 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:13.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:13.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:13.476 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:14.651 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0928a0) allocated [ CSSD]2013-05-23 16:03:14.652 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:03:14.653 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0928a0 - 1,2,3 [ CSSD]2013-05-23 16:03:14.653 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091c20) proc(0x7f431c0928a0) pid(29942) version 11:1:1:4 [ CSSD]2013-05-23 16:03:14.659 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0928a0), client(1/0x7f431c137d80) [ CSSD]2013-05-23 16:03:14.659 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137d80) [ CSSD]2013-05-23 16:03:14.659 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137d80 with con 0x7f431c137880, requested num -1 [ CSSD]2013-05-23 16:03:14.659 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:14.659 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1098 (0x7f431bfe1510) [ CSSD]2013-05-23 16:03:14.659 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1510 (RPC#1098) tag(44a002c) sent to node 1 [ CSSD]2013-05-23 16:03:14.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(044a002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:14.659 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014315, gin 1, member 0 [ CSSD]2013-05-23 16:03:14.659 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c081d70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:03:14.660 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:03:14.660 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:03:14.660 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1510 (RPC#1098) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:14.660 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1510, ret 0, client 0x7f431c137d80 member 0x7f431c137680 [ CSSD]2013-05-23 16:03:14.660 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137d80 [ CSSD]2013-05-23 16:03:14.660 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1098 [ CSSD]2013-05-23 16:03:14.666 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137d80) [ CSSD]2013-05-23 16:03:14.666 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137d80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:14.666 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137680) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:14.666 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1099 (0x7f431bfe1590) [ CSSD]2013-05-23 16:03:14.666 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1590 (RPC#1099) tag(44b002c) sent to node 1 [ CSSD]2013-05-23 16:03:14.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(044b002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:14.666 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:14.666 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1590 (RPC#1099) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:14.666 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1590, ret 0, client 0x7f431c137d80 member 0x7f431c137680 [ CSSD]2013-05-23 16:03:14.667 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1099 [ CSSD]2013-05-23 16:03:14.667 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137680) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:03:14.869 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0928a0 [ CSSD]2013-05-23 16:03:14.869 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0928a0) con(0x7f431c091c20) skgpid ospid 29942 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:03:15.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:15.391 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:15.482 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:16.740 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:16.740 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:17.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:17.396 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:17.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:17.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:17.594 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014316, gin 1, member 0 [ CSSD]2013-05-23 16:03:17.594 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:03:17.594 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:03:17.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:17.601 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:03:17.601 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:03:18.186 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:03:18.187 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1100 (0x7f431bfe1610) [ CSSD]2013-05-23 16:03:18.187 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1610 (RPC#1100) tag(44c002c) sent to node 1 [ CSSD]2013-05-23 16:03:18.187 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:18.187 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1100 (0x7f431bfe1610) [ CSSD]2013-05-23 16:03:18.187 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1610 (RPC#1100) state 6, flags 0x202 [ CSSD]2013-05-23 16:03:18.187 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1610, ret 0, client (nil) [ CSSD]2013-05-23 16:03:18.187 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1100 [ CSSD]2013-05-23 16:03:19.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:19.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:19.501 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:21.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:21.415 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:21.507 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:21.742 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:21.742 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:22.273 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1377b0) allocated [ CSSD]2013-05-23 16:03:22.274 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:03:22.274 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1377b0 - 1,2,3 [ CSSD]2013-05-23 16:03:22.274 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0926f0) proc(0x7f431c1377b0) pid(30001) version 11:1:1:4 [ CSSD]2013-05-23 16:03:22.280 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1377b0), client(1/0x7f431c137e20) [ CSSD]2013-05-23 16:03:22.280 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137e20) [ CSSD]2013-05-23 16:03:22.280 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137e20 with con 0x7f431c092930, requested num -1 [ CSSD]2013-05-23 16:03:22.280 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:22.280 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1101 (0x7f431bfe1690) [ CSSD]2013-05-23 16:03:22.280 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1690 (RPC#1101) tag(44d002c) sent to node 1 [ CSSD]2013-05-23 16:03:22.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(044d002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:22.280 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014317, gin 1, member 0 [ CSSD]2013-05-23 16:03:22.280 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c081e40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:03:22.281 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:03:22.281 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:03:22.281 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1690 (RPC#1101) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:22.281 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1690, ret 0, client 0x7f431c137e20 member 0x7f431c1376a0 [ CSSD]2013-05-23 16:03:22.281 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137e20 [ CSSD]2013-05-23 16:03:22.281 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1101 [ CSSD]2013-05-23 16:03:22.287 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137e20) [ CSSD]2013-05-23 16:03:22.287 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137e20), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:22.287 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1376a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:22.287 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1102 (0x7f431bfe1710) [ CSSD]2013-05-23 16:03:22.288 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1710 (RPC#1102) tag(44e002c) sent to node 1 [ CSSD]2013-05-23 16:03:22.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(044e002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:22.288 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:22.288 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1710 (RPC#1102) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:22.288 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1710, ret 0, client 0x7f431c137e20 member 0x7f431c1376a0 [ CSSD]2013-05-23 16:03:22.288 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1102 [ CSSD]2013-05-23 16:03:22.288 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1376a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:03:22.490 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1377b0 [ CSSD]2013-05-23 16:03:22.491 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1377b0) con(0x7f431c0926f0) skgpid ospid 30001 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:03:23.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:23.421 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:23.558 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:25.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:25.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:25.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:26.746 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:26.746 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:27.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:27.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:27.656 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:28.216 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:03:28.216 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1103 (0x7f431bfe1790) [ CSSD]2013-05-23 16:03:28.216 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1790 (RPC#1103) tag(44f002c) sent to node 1 [ CSSD]2013-05-23 16:03:28.217 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:28.217 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1103 (0x7f431bfe1790) [ CSSD]2013-05-23 16:03:28.217 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1790 (RPC#1103) state 6, flags 0x202 [ CSSD]2013-05-23 16:03:28.217 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1790, ret 0, client (nil) [ CSSD]2013-05-23 16:03:28.217 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1103 [ CSSD]2013-05-23 16:03:29.498 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:29.498 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:29.662 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:30.748 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:30.748 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:03:31.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:31.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:31.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:32.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:32.859 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014318, gin 1, member 0 [ CSSD]2013-05-23 16:03:32.859 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:03:32.859 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:03:32.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:32.865 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:03:32.865 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:03:33.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:33.510 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:33.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:35.542 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:35.542 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:35.721 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:35.751 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:35.751 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:37.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:37.168 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014319, gin 1, member 0 [ CSSD]2013-05-23 16:03:37.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:03:37.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:03:37.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:37.175 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:03:37.175 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:03:37.303 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:03:37.304 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:03:37.306 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:03:37.307 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:03:37.308 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:03:37.309 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:03:37.522 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092850) allocated [ CSSD]2013-05-23 16:03:37.523 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:03:37.523 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092850 - 1,2,3 [ CSSD]2013-05-23 16:03:37.523 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091b50) proc(0x7f431c092850) pid(30141) version 11:1:1:4 [ CSSD]2013-05-23 16:03:37.539 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c092850), client(1/0x7f431c137da0) [ CSSD]2013-05-23 16:03:37.539 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137da0) [ CSSD]2013-05-23 16:03:37.539 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137da0 with con 0x7f431c1378a0, requested num -1 [ CSSD]2013-05-23 16:03:37.539 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:37.539 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1104 (0x7f431bfe1810) [ CSSD]2013-05-23 16:03:37.539 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1810 (RPC#1104) tag(450002c) sent to node 1 [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0450002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014320, gin 1, member 0 [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c081d70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1810 (RPC#1104) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1810, ret 0, client 0x7f431c137da0 member 0x7f431c1376a0 [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137da0 [ CSSD]2013-05-23 16:03:37.540 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1104 [ CSSD]2013-05-23 16:03:37.547 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137da0) [ CSSD]2013-05-23 16:03:37.547 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137da0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:37.547 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1376a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:37.547 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1105 (0x7f431bfe1890) [ CSSD]2013-05-23 16:03:37.547 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1890 (RPC#1105) tag(451002c) sent to node 1 [ CSSD]2013-05-23 16:03:37.547 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0451002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:37.547 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:37.547 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1890 (RPC#1105) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:37.547 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1890, ret 0, client 0x7f431c137da0 member 0x7f431c1376a0 [ CSSD]2013-05-23 16:03:37.548 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1105 [ CSSD]2013-05-23 16:03:37.548 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1376a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:03:37.588 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:37.588 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:37.750 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092850 [ CSSD]2013-05-23 16:03:37.750 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092850) con(0x7f431c091b50) skgpid ospid 30141 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:03:37.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:38.246 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:03:38.246 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1106 (0x7f431bfe1910) [ CSSD]2013-05-23 16:03:38.246 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1910 (RPC#1106) tag(452002c) sent to node 1 [ CSSD]2013-05-23 16:03:38.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:38.247 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1106 (0x7f431bfe1910) [ CSSD]2013-05-23 16:03:38.247 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1910 (RPC#1106) state 6, flags 0x202 [ CSSD]2013-05-23 16:03:38.247 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1910, ret 0, client (nil) [ CSSD]2013-05-23 16:03:38.247 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1106 [ CSSD]2013-05-23 16:03:39.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:39.599 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:39.763 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:40.754 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:40.755 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:41.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:41.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:41.768 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:43.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:43.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:43.779 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:45.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:45.622 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:45.757 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:45.758 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:45.805 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:47.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:47.628 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:47.857 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:48.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:48.106 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014321, gin 1, member 0 [ CSSD]2013-05-23 16:03:48.106 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:03:48.106 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:03:48.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:48.113 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:03:48.113 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:03:48.266 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:03:48.266 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1107 (0x7f431bfe1990) [ CSSD]2013-05-23 16:03:48.267 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1990 (RPC#1107) tag(453002c) sent to node 1 [ CSSD]2013-05-23 16:03:48.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:48.267 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1107 (0x7f431bfe1990) [ CSSD]2013-05-23 16:03:48.267 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1990 (RPC#1107) state 6, flags 0x202 [ CSSD]2013-05-23 16:03:48.267 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1990, ret 0, client (nil) [ CSSD]2013-05-23 16:03:48.267 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1107 [ CSSD]2013-05-23 16:03:49.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:49.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:49.863 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:50.761 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:50.761 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:51.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:51.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:51.868 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:52.782 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1377b0) allocated [ CSSD]2013-05-23 16:03:52.783 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:03:52.784 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1377b0 - 1,2,3 [ CSSD]2013-05-23 16:03:52.784 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091c40) proc(0x7f431c1377b0) pid(30312) version 11:1:1:4 [ CSSD]2013-05-23 16:03:52.789 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1377b0), client(1/0x7f431c137e60) [ CSSD]2013-05-23 16:03:52.789 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137e60) [ CSSD]2013-05-23 16:03:52.789 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137e60 with con 0x7f431c0928c0, requested num -1 [ CSSD]2013-05-23 16:03:52.789 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:52.789 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1108 (0x7f431bfe1a10) [ CSSD]2013-05-23 16:03:52.789 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1a10 (RPC#1108) tag(454002c) sent to node 1 [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0454002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014322, gin 1, member 0 [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c081e40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1a10 (RPC#1108) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1a10, ret 0, client 0x7f431c137e60 member 0x7f431c1376a0 [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137e60 [ CSSD]2013-05-23 16:03:52.790 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1108 [ CSSD]2013-05-23 16:03:52.796 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137e60) [ CSSD]2013-05-23 16:03:52.796 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137e60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:52.796 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1376a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:03:52.796 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1109 (0x7f431bfe1a90) [ CSSD]2013-05-23 16:03:52.796 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1a90 (RPC#1109) tag(455002c) sent to node 1 [ CSSD]2013-05-23 16:03:52.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0455002c) incarnation(174631051) [ CSSD]2013-05-23 16:03:52.797 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:03:52.797 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1a90 (RPC#1109) state 6, flags 0x100 [ CSSD]2013-05-23 16:03:52.797 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1a90, ret 0, client 0x7f431c137e60 member 0x7f431c1376a0 [ CSSD]2013-05-23 16:03:52.797 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1109 [ CSSD]2013-05-23 16:03:52.797 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1376a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:03:52.999 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1377b0 [ CSSD]2013-05-23 16:03:52.999 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1377b0) con(0x7f431c091c40) skgpid ospid 30312 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:03:53.737 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:53.737 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:53.879 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:55.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:55.743 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:55.764 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:03:55.764 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:03:55.905 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:57.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:57.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:57.957 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:03:58.286 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:03:58.286 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1110 (0x7f431bfe1b10) [ CSSD]2013-05-23 16:03:58.286 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1b10 (RPC#1110) tag(456002c) sent to node 1 [ CSSD]2013-05-23 16:03:58.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:58.287 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1110 (0x7f431bfe1b10) [ CSSD]2013-05-23 16:03:58.287 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1b10 (RPC#1110) state 6, flags 0x202 [ CSSD]2013-05-23 16:03:58.287 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1b10, ret 0, client (nil) [ CSSD]2013-05-23 16:03:58.287 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1110 [ CSSD]2013-05-23 16:03:59.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:03:59.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:03:59.963 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:00.767 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:00.767 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:01.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:01.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:01.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:03.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:03.364 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014323, gin 1, member 0 [ CSSD]2013-05-23 16:04:03.364 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:04:03.364 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:04:03.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:03.371 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:04:03.371 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:04:03.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:03.898 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:03.999 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:05.770 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:05.770 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:05.783 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092880) allocated [ CSSD]2013-05-23 16:04:05.784 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:04:05.784 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092880 - 1,2,3 [ CSSD]2013-05-23 16:04:05.784 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091c00) proc(0x7f431c092880) pid(30361/30361) version 11:1:1:4 [ CSSD]2013-05-23 16:04:05.800 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c092880), client(1/0x7f431c137c20) [ CSSD]2013-05-23 16:04:05.800 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c137c20) [ CSSD]2013-05-23 16:04:05.800 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:04:05.800 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:04:05.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:05.903 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:06.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:06.688 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c092880), client(2/0x7f431c081f70) [ CSSD]2013-05-23 16:04:06.688 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c081f70) [ CSSD]2013-05-23 16:04:06.688 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:04:06.688 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 16:04:07.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:07.928 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:08.041 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ad10) allocated [ CSSD]2013-05-23 16:04:08.042 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:04:08.042 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ad10 - 1,2,3 [ CSSD]2013-05-23 16:04:08.042 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0915c0) proc(0x7f431c13ad10) pid(30466) version 11:1:1:4 [ CSSD]2013-05-23 16:04:08.048 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ad10), client(1/0x7f431c13b490) [ CSSD]2013-05-23 16:04:08.048 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13b490) [ CSSD]2013-05-23 16:04:08.048 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13b490 with con 0x7f431c13af90, requested num -1 [ CSSD]2013-05-23 16:04:08.048 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:08.048 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1111 (0x7f431bfe1b90) [ CSSD]2013-05-23 16:04:08.048 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1b90 (RPC#1111) tag(457002c) sent to node 1 [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0457002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014324, gin 1, member 0 [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13b600/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1b90 (RPC#1111) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1b90, ret 0, client 0x7f431c13b490 member 0x7f431c13ac00 [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13b490 [ CSSD]2013-05-23 16:04:08.049 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1111 [ CSSD]2013-05-23 16:04:08.055 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13b490) [ CSSD]2013-05-23 16:04:08.055 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13b490), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:08.055 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ac00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:08.055 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1112 (0x7f431bfe1c10) [ CSSD]2013-05-23 16:04:08.055 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1c10 (RPC#1112) tag(458002c) sent to node 1 [ CSSD]2013-05-23 16:04:08.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0458002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:08.055 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:08.055 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1c10 (RPC#1112) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:08.055 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1c10, ret 0, client 0x7f431c13b490 member 0x7f431c13ac00 [ CSSD]2013-05-23 16:04:08.056 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1112 [ CSSD]2013-05-23 16:04:08.056 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13ac00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:04:08.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:08.258 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ad10 [ CSSD]2013-05-23 16:04:08.258 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ad10) con(0x7f431c0915c0) skgpid ospid 30466 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:04:08.316 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:04:08.317 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1113 (0x7f431bfe1c90) [ CSSD]2013-05-23 16:04:08.317 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1c90 (RPC#1113) tag(459002c) sent to node 1 [ CSSD]2013-05-23 16:04:08.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:08.317 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1113 (0x7f431bfe1c90) [ CSSD]2013-05-23 16:04:08.317 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1c90 (RPC#1113) state 6, flags 0x202 [ CSSD]2013-05-23 16:04:08.317 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1c90, ret 0, client (nil) [ CSSD]2013-05-23 16:04:08.318 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1113 [ CSSD]2013-05-23 16:04:10.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:10.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:10.063 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:10.773 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:10.773 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:11.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:11.119 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014325, gin 1, member 0 [ CSSD]2013-05-23 16:04:11.119 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:04:11.119 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:04:11.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:11.126 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:04:11.126 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:04:12.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:12.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:12.089 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:14.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:14.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:14.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:14.776 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:14.776 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:04:14.910 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ade0) allocated [ CSSD]2013-05-23 16:04:14.911 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:04:14.912 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ade0 - 1,2,3 [ CSSD]2013-05-23 16:04:14.912 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c082670) proc(0x7f431c13ade0) pid(30554) version 11:1:1:4 [ CSSD]2013-05-23 16:04:14.928 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ade0), client(1/0x7f431c083ae0) [ CSSD]2013-05-23 16:04:14.928 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c083ae0) [ CSSD]2013-05-23 16:04:14.928 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c083ae0 with con 0x7f431c13b0d0, requested num -1 [ CSSD]2013-05-23 16:04:14.928 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:14.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1114 (0x7f431bfe1d10) [ CSSD]2013-05-23 16:04:14.929 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1d10 (RPC#1114) tag(45a002c) sent to node 1 [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(045a002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014326, gin 1, member 0 [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13b6a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1d10 (RPC#1114) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1d10, ret 0, client 0x7f431c083ae0 member 0x7f431c13acd0 [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c083ae0 [ CSSD]2013-05-23 16:04:14.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1114 [ CSSD]2013-05-23 16:04:14.936 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c083ae0) [ CSSD]2013-05-23 16:04:14.936 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c083ae0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:14.936 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13acd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:14.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1115 (0x7f431bfe1d90) [ CSSD]2013-05-23 16:04:14.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1d90 (RPC#1115) tag(45b002c) sent to node 1 [ CSSD]2013-05-23 16:04:14.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(045b002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:14.936 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:14.936 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1d90 (RPC#1115) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:14.936 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1d90, ret 0, client 0x7f431c083ae0 member 0x7f431c13acd0 [ CSSD]2013-05-23 16:04:14.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1115 [ CSSD]2013-05-23 16:04:14.937 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13acd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:04:15.139 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ade0 [ CSSD]2013-05-23 16:04:15.139 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ade0) con(0x7f431c082670) skgpid ospid 30554 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:04:16.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:16.044 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:16.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:18.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:18.050 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:18.158 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:18.336 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:04:18.337 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1116 (0x7f431bfe1e10) [ CSSD]2013-05-23 16:04:18.337 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1e10 (RPC#1116) tag(45c002c) sent to node 1 [ CSSD]2013-05-23 16:04:18.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:18.338 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1116 (0x7f431bfe1e10) [ CSSD]2013-05-23 16:04:18.338 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1e10 (RPC#1116) state 6, flags 0x202 [ CSSD]2013-05-23 16:04:18.338 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1e10, ret 0, client (nil) [ CSSD]2013-05-23 16:04:18.338 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1116 [ CSSD]2013-05-23 16:04:18.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:18.629 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014327, gin 1, member 0 [ CSSD]2013-05-23 16:04:18.629 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:04:18.629 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:04:18.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:18.636 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:04:18.636 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:04:19.779 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:19.779 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:20.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:20.101 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:20.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:22.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:22.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:22.190 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:23.291 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ade0) allocated [ CSSD]2013-05-23 16:04:23.293 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:04:23.293 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ade0 - 1,2,3 [ CSSD]2013-05-23 16:04:23.293 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c082670) proc(0x7f431c13ade0) pid(30613) version 11:1:1:4 [ CSSD]2013-05-23 16:04:23.299 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ade0), client(1/0x7f431c083ae0) [ CSSD]2013-05-23 16:04:23.299 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c083ae0) [ CSSD]2013-05-23 16:04:23.299 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c083ae0 with con 0x7f431c13b0d0, requested num -1 [ CSSD]2013-05-23 16:04:23.299 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:23.299 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1117 (0x7f431bfe1e90) [ CSSD]2013-05-23 16:04:23.299 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1e90 (RPC#1117) tag(45d002c) sent to node 1 [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(045d002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014328, gin 1, member 0 [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13b6a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1e90 (RPC#1117) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1e90, ret 0, client 0x7f431c083ae0 member 0x7f431c13acd0 [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c083ae0 [ CSSD]2013-05-23 16:04:23.300 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1117 [ CSSD]2013-05-23 16:04:23.306 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c083ae0) [ CSSD]2013-05-23 16:04:23.306 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c083ae0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:23.306 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13acd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:23.306 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1118 (0x7f431bfe1f10) [ CSSD]2013-05-23 16:04:23.306 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1f10 (RPC#1118) tag(45e002c) sent to node 1 [ CSSD]2013-05-23 16:04:23.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(045e002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:23.306 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:23.306 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1f10 (RPC#1118) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:23.306 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1f10, ret 0, client 0x7f431c083ae0 member 0x7f431c13acd0 [ CSSD]2013-05-23 16:04:23.307 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1118 [ CSSD]2013-05-23 16:04:23.307 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13acd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:04:23.509 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ade0 [ CSSD]2013-05-23 16:04:23.509 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ade0) con(0x7f431c082670) skgpid ospid 30613 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:04:24.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:24.119 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:24.152 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:04:24.153 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:04:24.155 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:04:24.155 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:04:24.201 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:24.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:24.233 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014329, gin 1, member 0 [ CSSD]2013-05-23 16:04:24.233 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:04:24.233 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:04:24.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:24.239 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:04:24.239 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:04:24.782 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:24.782 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:26.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:26.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:26.206 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:28.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:28.129 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:28.258 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:28.357 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:04:28.357 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1119 (0x7f431bfe1f90) [ CSSD]2013-05-23 16:04:28.357 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1f90 (RPC#1119) tag(45f002c) sent to node 1 [ CSSD]2013-05-23 16:04:28.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:28.358 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1119 (0x7f431bfe1f90) [ CSSD]2013-05-23 16:04:28.358 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1f90 (RPC#1119) state 6, flags 0x202 [ CSSD]2013-05-23 16:04:28.358 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1f90, ret 0, client (nil) [ CSSD]2013-05-23 16:04:28.358 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1119 [ CSSD]2013-05-23 16:04:29.785 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:29.785 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:30.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:30.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:30.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:32.227 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:32.227 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:32.307 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:33.787 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:33.787 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:04:33.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:33.880 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014330, gin 1, member 0 [ CSSD]2013-05-23 16:04:33.880 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:04:33.880 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:04:33.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:33.887 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:04:33.887 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:04:34.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:34.219 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:34.332 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:36.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:36.244 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:36.356 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:38.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:38.250 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:38.369 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:38.397 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:04:38.397 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1120 (0x7f431bfe2010) [ CSSD]2013-05-23 16:04:38.397 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2010 (RPC#1120) tag(460002c) sent to node 1 [ CSSD]2013-05-23 16:04:38.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:38.398 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1120 (0x7f431bfe2010) [ CSSD]2013-05-23 16:04:38.398 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2010 (RPC#1120) state 6, flags 0x202 [ CSSD]2013-05-23 16:04:38.398 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2010, ret 0, client (nil) [ CSSD]2013-05-23 16:04:38.398 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1120 [ CSSD]2013-05-23 16:04:38.541 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ad10) allocated [ CSSD]2013-05-23 16:04:38.542 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:04:38.542 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ad10 - 1,2,3 [ CSSD]2013-05-23 16:04:38.542 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0915c0) proc(0x7f431c13ad10) pid(30754) version 11:1:1:4 [ CSSD]2013-05-23 16:04:38.548 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ad10), client(1/0x7f431c13b490) [ CSSD]2013-05-23 16:04:38.548 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13b490) [ CSSD]2013-05-23 16:04:38.549 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13b490 with con 0x7f431c13af90, requested num -1 [ CSSD]2013-05-23 16:04:38.549 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:38.549 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1121 (0x7f431bfe2090) [ CSSD]2013-05-23 16:04:38.549 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2090 (RPC#1121) tag(461002c) sent to node 1 [ CSSD]2013-05-23 16:04:38.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0461002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:38.549 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014331, gin 1, member 0 [ CSSD]2013-05-23 16:04:38.549 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13b600/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:04:38.549 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:04:38.550 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:04:38.550 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2090 (RPC#1121) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:38.550 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2090, ret 0, client 0x7f431c13b490 member 0x7f431c13ac00 [ CSSD]2013-05-23 16:04:38.550 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13b490 [ CSSD]2013-05-23 16:04:38.550 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1121 [ CSSD]2013-05-23 16:04:38.556 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13b490) [ CSSD]2013-05-23 16:04:38.556 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13b490), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:38.556 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ac00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:38.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1122 (0x7f431bfe2110) [ CSSD]2013-05-23 16:04:38.556 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2110 (RPC#1122) tag(462002c) sent to node 1 [ CSSD]2013-05-23 16:04:38.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0462002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:38.557 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:38.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2110 (RPC#1122) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:38.557 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2110, ret 0, client 0x7f431c13b490 member 0x7f431c13ac00 [ CSSD]2013-05-23 16:04:38.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1122 [ CSSD]2013-05-23 16:04:38.557 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13ac00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:04:38.759 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ad10 [ CSSD]2013-05-23 16:04:38.759 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ad10) con(0x7f431c0915c0) skgpid ospid 30754 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:04:38.790 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:38.791 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:40.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:40.252 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:40.375 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:42.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:42.278 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:42.386 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:43.794 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:43.794 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:44.310 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:44.310 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:44.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:46.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:46.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:46.417 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:48.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:48.321 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:48.427 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:04:48.427 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1123 (0x7f431bfe2190) [ CSSD]2013-05-23 16:04:48.428 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2190 (RPC#1123) tag(463002c) sent to node 1 [ CSSD]2013-05-23 16:04:48.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:48.428 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1123 (0x7f431bfe2190) [ CSSD]2013-05-23 16:04:48.428 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2190 (RPC#1123) state 6, flags 0x202 [ CSSD]2013-05-23 16:04:48.428 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2190, ret 0, client (nil) [ CSSD]2013-05-23 16:04:48.428 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1123 [ CSSD]2013-05-23 16:04:48.469 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:48.797 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:48.797 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:49.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:49.140 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014332, gin 1, member 0 [ CSSD]2013-05-23 16:04:49.140 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:04:49.140 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:04:49.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:49.147 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:04:49.147 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:04:50.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:50.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:50.476 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:52.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:52.358 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:52.487 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:52.800 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:52.800 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:04:53.791 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ade0) allocated [ CSSD]2013-05-23 16:04:53.792 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:04:53.793 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ade0 - 1,2,3 [ CSSD]2013-05-23 16:04:53.793 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c082670) proc(0x7f431c13ade0) pid(30906) version 11:1:1:4 [ CSSD]2013-05-23 16:04:53.809 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ade0), client(1/0x7f431c083ae0) [ CSSD]2013-05-23 16:04:53.809 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c083ae0) [ CSSD]2013-05-23 16:04:53.809 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c083ae0 with con 0x7f431c13b0d0, requested num -1 [ CSSD]2013-05-23 16:04:53.809 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:53.809 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1124 (0x7f431bfe2210) [ CSSD]2013-05-23 16:04:53.809 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2210 (RPC#1124) tag(464002c) sent to node 1 [ CSSD]2013-05-23 16:04:53.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0464002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:53.809 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014333, gin 1, member 0 [ CSSD]2013-05-23 16:04:53.809 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13b6a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:04:53.810 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:04:53.810 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:04:53.810 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2210 (RPC#1124) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:53.810 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2210, ret 0, client 0x7f431c083ae0 member 0x7f431c13acd0 [ CSSD]2013-05-23 16:04:53.810 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c083ae0 [ CSSD]2013-05-23 16:04:53.810 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1124 [ CSSD]2013-05-23 16:04:53.816 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c083ae0) [ CSSD]2013-05-23 16:04:53.816 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c083ae0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:53.816 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13acd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:04:53.816 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1125 (0x7f431bfe2290) [ CSSD]2013-05-23 16:04:53.816 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2290 (RPC#1125) tag(465002c) sent to node 1 [ CSSD]2013-05-23 16:04:53.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0465002c) incarnation(174631051) [ CSSD]2013-05-23 16:04:53.817 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:04:53.817 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2290 (RPC#1125) state 6, flags 0x100 [ CSSD]2013-05-23 16:04:53.817 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2290, ret 0, client 0x7f431c083ae0 member 0x7f431c13acd0 [ CSSD]2013-05-23 16:04:53.817 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1125 [ CSSD]2013-05-23 16:04:53.817 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13acd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:04:54.019 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ade0 [ CSSD]2013-05-23 16:04:54.019 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ade0) con(0x7f431c082670) skgpid ospid 30906 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:04:54.430 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:54.430 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:54.512 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:56.436 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:56.436 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:56.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:04:57.803 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:04:57.803 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:04:58.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:58.442 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:04:58.447 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:04:58.447 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1126 (0x7f431bfe2310) [ CSSD]2013-05-23 16:04:58.448 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2310 (RPC#1126) tag(466002c) sent to node 1 [ CSSD]2013-05-23 16:04:58.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:04:58.448 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1126 (0x7f431bfe2310) [ CSSD]2013-05-23 16:04:58.448 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2310 (RPC#1126) state 6, flags 0x202 [ CSSD]2013-05-23 16:04:58.448 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2310, ret 0, client (nil) [ CSSD]2013-05-23 16:04:58.448 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1126 [ CSSD]2013-05-23 16:04:58.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:00.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:00.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:00.576 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:02.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:02.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:02.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:02.806 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:02.806 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:04.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:04.404 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014334, gin 1, member 0 [ CSSD]2013-05-23 16:05:04.404 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:05:04.404 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:05:04.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:04.411 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:05:04.411 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:05:04.433 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ad10) allocated [ CSSD]2013-05-23 16:05:04.434 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:05:04.435 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ad10 - 1,2,3 [ CSSD]2013-05-23 16:05:04.435 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0915c0) proc(0x7f431c13ad10) pid(29667/29667) version 11:1:1:4 [ CSSD]2013-05-23 16:05:04.451 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ad10), client(1/0x7f431c13b2f0) [ CSSD]2013-05-23 16:05:04.451 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c13b2f0) [ CSSD]2013-05-23 16:05:04.451 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:05:04.451 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:05:04.473 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ad10), client(2/0x7f431c13b7e0) [ CSSD]2013-05-23 16:05:04.473 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c13b7e0) [ CSSD]2013-05-23 16:05:04.473 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:05:04.473 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 16:05:04.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:04.511 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:04.529 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1504a0) allocated [ CSSD]2013-05-23 16:05:04.530 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:05:04.531 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1504a0 - 1,2,3 [ CSSD]2013-05-23 16:05:04.531 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13bee0) proc(0x7f431c1504a0) pid(30359/30359) version 11:1:1:4 [ CSSD]2013-05-23 16:05:04.537 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1504a0), client(1/0x7f431c1509e0) [ CSSD]2013-05-23 16:05:04.537 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c1509e0) [ CSSD]2013-05-23 16:05:04.537 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:05:04.537 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:05:04.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:04.673 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c143f20) allocated [ CSSD]2013-05-23 16:05:04.675 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:05:04.675 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c143f20 - 1,2,3 [ CSSD]2013-05-23 16:05:04.675 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c141ca0) proc(0x7f431c143f20) pid(30357/30357) version 11:1:1:4 [ CSSD]2013-05-23 16:05:04.691 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c143f20), client(1/0x7f431c144540) [ CSSD]2013-05-23 16:05:04.691 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c144540) [ CSSD]2013-05-23 16:05:04.691 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:05:04.692 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 16:05:04.692 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1504a0), client(2/0x7f431c144a40) [ CSSD]2013-05-23 16:05:04.692 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c144a40) [ CSSD]2013-05-23 16:05:04.692 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:05:04.692 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:05:04.764 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c143f20), client(2/0x7f431c144f40) [ CSSD]2013-05-23 16:05:04.764 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c144f40) [ CSSD]2013-05-23 16:05:04.764 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:05:04.764 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:05:06.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:06.535 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:06.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:07.810 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:07.810 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:08.468 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:05:08.469 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1127 (0x7f431bfe2390) [ CSSD]2013-05-23 16:05:08.469 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2390 (RPC#1127) tag(467002c) sent to node 1 [ CSSD]2013-05-23 16:05:08.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:08.469 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1127 (0x7f431bfe2390) [ CSSD]2013-05-23 16:05:08.469 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2390 (RPC#1127) state 6, flags 0x202 [ CSSD]2013-05-23 16:05:08.469 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2390, ret 0, client (nil) [ CSSD]2013-05-23 16:05:08.470 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1127 [ CSSD]2013-05-23 16:05:08.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:08.540 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:08.544 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147880) allocated [ CSSD]2013-05-23 16:05:08.545 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:05:08.545 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147880 - 1,2,3 [ CSSD]2013-05-23 16:05:08.545 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c150b50) proc(0x7f431c147880) pid(31059) version 11:1:1:4 [ CSSD]2013-05-23 16:05:08.670 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:09.053 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c14a440) allocated [ CSSD]2013-05-23 16:05:09.053 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:05:09.054 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c14a440 - 1,2,3 [ CSSD]2013-05-23 16:05:09.054 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1481b0) proc(0x7f431c14a440) pid(31151) version 11:1:1:4 [ CSSD]2013-05-23 16:05:09.060 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c14a440), client(1/0x7f431c14a3a0) [ CSSD]2013-05-23 16:05:09.060 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14a3a0) [ CSSD]2013-05-23 16:05:09.060 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14a3a0 with con 0x7f431c14a710, requested num -1 [ CSSD]2013-05-23 16:05:09.060 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:09.060 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1128 (0x7f431bfe2410) [ CSSD]2013-05-23 16:05:09.060 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2410 (RPC#1128) tag(468002c) sent to node 1 [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0468002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014335, gin 1, member 0 [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14ad00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2410 (RPC#1128) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2410, ret 0, client 0x7f431c14a3a0 member 0x7f431c14b000 [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14a3a0 [ CSSD]2013-05-23 16:05:09.061 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1128 [ CSSD]2013-05-23 16:05:09.079 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14a3a0) [ CSSD]2013-05-23 16:05:09.079 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14a3a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:09.079 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c14b000) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:09.079 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1129 (0x7f431bfe2490) [ CSSD]2013-05-23 16:05:09.079 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2490 (RPC#1129) tag(469002c) sent to node 1 [ CSSD]2013-05-23 16:05:09.079 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0469002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:09.079 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:09.079 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2490 (RPC#1129) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:09.079 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2490, ret 0, client 0x7f431c14a3a0 member 0x7f431c14b000 [ CSSD]2013-05-23 16:05:09.080 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1129 [ CSSD]2013-05-23 16:05:09.080 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c14b000) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:05:09.205 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147880 [ CSSD]2013-05-23 16:05:09.205 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147880) con(0x7f431c150b50) skgpid ospid 31059 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:09.282 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c14a440 [ CSSD]2013-05-23 16:05:09.282 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c14a440) con(0x7f431c1481b0) skgpid ospid 31151 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:10.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:10.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:10.676 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:11.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:11.377 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014336, gin 1, member 0 [ CSSD]2013-05-23 16:05:11.377 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:05:11.377 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:05:11.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:11.385 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:05:11.385 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:05:11.812 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:11.813 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:05:12.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:12.558 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:12.689 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:14.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:14.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:14.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:15.174 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1478b0) allocated [ CSSD]2013-05-23 16:05:15.175 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:05:15.176 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1478b0 - 1,2,3 [ CSSD]2013-05-23 16:05:15.176 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c145620) proc(0x7f431c1478b0) pid(31244) version 11:1:1:4 [ CSSD]2013-05-23 16:05:15.183 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1478b0), client(1/0x7f431c147810) [ CSSD]2013-05-23 16:05:15.183 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c147810) [ CSSD]2013-05-23 16:05:15.183 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c147810 with con 0x7f431c147b80, requested num -1 [ CSSD]2013-05-23 16:05:15.183 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:15.183 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1130 (0x7f431bfe2510) [ CSSD]2013-05-23 16:05:15.183 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2510 (RPC#1130) tag(46a002c) sent to node 1 [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(046a002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014337, gin 1, member 0 [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c148170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2510 (RPC#1130) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2510, ret 0, client 0x7f431c147810 member 0x7f431c148470 [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c147810 [ CSSD]2013-05-23 16:05:15.184 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1130 [ CSSD]2013-05-23 16:05:15.190 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c147810) [ CSSD]2013-05-23 16:05:15.190 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c147810), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:15.190 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c148470) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:15.190 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1131 (0x7f431bfe2590) [ CSSD]2013-05-23 16:05:15.190 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2590 (RPC#1131) tag(46b002c) sent to node 1 [ CSSD]2013-05-23 16:05:15.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(046b002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:15.191 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:15.191 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2590 (RPC#1131) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:15.191 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2590, ret 0, client 0x7f431c147810 member 0x7f431c148470 [ CSSD]2013-05-23 16:05:15.191 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1131 [ CSSD]2013-05-23 16:05:15.191 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c148470) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:05:15.393 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1478b0 [ CSSD]2013-05-23 16:05:15.393 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1478b0) con(0x7f431c145620) skgpid ospid 31244 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:16.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:16.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:16.720 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:16.816 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:16.816 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:18.498 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:05:18.498 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1132 (0x7f431bfe2610) [ CSSD]2013-05-23 16:05:18.499 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2610 (RPC#1132) tag(46c002c) sent to node 1 [ CSSD]2013-05-23 16:05:18.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:18.499 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1132 (0x7f431bfe2610) [ CSSD]2013-05-23 16:05:18.499 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2610 (RPC#1132) state 6, flags 0x202 [ CSSD]2013-05-23 16:05:18.499 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2610, ret 0, client (nil) [ CSSD]2013-05-23 16:05:18.500 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1132 [ CSSD]2013-05-23 16:05:18.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:18.640 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:18.792 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:18.925 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:05:18.925 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:05:18.925 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:05:18.925 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:05:18.930 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:18.930 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:05:18.930 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:05:18.930 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 16:05:18.931 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:05:18.931 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0840c0) skgpid 26011 ospid 26011 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:19.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:19.659 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014338, gin 1, member 0 [ CSSD]2013-05-23 16:05:19.659 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:05:19.659 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:05:19.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:19.666 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:05:19.666 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:05:20.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:20.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:20.797 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:21.819 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:21.819 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:22.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:22.759 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:22.808 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:24.314 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:05:24.315 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:05:24.316 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:05:24.316 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c108370) pid(31321) version 11:1:1:4 [ CSSD]2013-05-23 16:05:24.322 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:05:24.322 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:24.322 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c150b50, requested num -1 [ CSSD]2013-05-23 16:05:24.322 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:24.322 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1133 (0x7f431bfe2690) [ CSSD]2013-05-23 16:05:24.322 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2690 (RPC#1133) tag(46d002c) sent to node 1 [ CSSD]2013-05-23 16:05:24.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(046d002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:24.322 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014339, gin 1, member 0 [ CSSD]2013-05-23 16:05:24.322 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1450b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:05:24.323 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:05:24.323 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:05:24.323 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2690 (RPC#1133) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:24.323 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2690, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 16:05:24.323 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:05:24.323 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1133 [ CSSD]2013-05-23 16:05:24.329 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:24.329 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:24.329 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:24.329 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1134 (0x7f431bfe2710) [ CSSD]2013-05-23 16:05:24.329 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2710 (RPC#1134) tag(46e002c) sent to node 1 [ CSSD]2013-05-23 16:05:24.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(046e002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:24.330 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:24.330 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2710 (RPC#1134) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:24.330 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2710, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 16:05:24.330 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1134 [ CSSD]2013-05-23 16:05:24.330 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:05:24.533 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:05:24.533 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0840c0) skgpid ospid 31321 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:24.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:24.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:24.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:26.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:26.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:26.822 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:26.822 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:26.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:28.518 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:05:28.518 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1135 (0x7f431bfe2790) [ CSSD]2013-05-23 16:05:28.519 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2790 (RPC#1135) tag(46f002c) sent to node 1 [ CSSD]2013-05-23 16:05:28.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:28.519 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1135 (0x7f431bfe2790) [ CSSD]2013-05-23 16:05:28.519 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2790 (RPC#1135) state 6, flags 0x202 [ CSSD]2013-05-23 16:05:28.519 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2790, ret 0, client (nil) [ CSSD]2013-05-23 16:05:28.519 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1135 [ CSSD]2013-05-23 16:05:28.829 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:28.829 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:28.871 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:30.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:30.835 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:30.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:31.826 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:31.826 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:32.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:32.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:32.890 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:34.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:34.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:34.910 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014340, gin 1, member 0 [ CSSD]2013-05-23 16:05:34.910 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:05:34.910 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:05:34.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:34.950 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:05:34.950 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:05:34.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:34.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:36.829 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:36.830 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:36.920 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:36.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:36.937 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:38.538 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:05:38.538 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1136 (0x7f431bfe2810) [ CSSD]2013-05-23 16:05:38.539 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2810 (RPC#1136) tag(470002c) sent to node 1 [ CSSD]2013-05-23 16:05:38.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:38.539 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1136 (0x7f431bfe2810) [ CSSD]2013-05-23 16:05:38.539 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2810 (RPC#1136) state 6, flags 0x202 [ CSSD]2013-05-23 16:05:38.539 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2810, ret 0, client (nil) [ CSSD]2013-05-23 16:05:38.539 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1136 [ CSSD]2013-05-23 16:05:38.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:38.949 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:38.971 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:39.565 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:05:39.566 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:05:39.567 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:05:39.567 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(31511) version 11:1:1:4 [ CSSD]2013-05-23 16:05:39.573 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:05:39.573 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:39.573 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 16:05:39.573 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:39.573 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1137 (0x7f431bfe2890) [ CSSD]2013-05-23 16:05:39.573 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2890 (RPC#1137) tag(471002c) sent to node 1 [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0471002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014341, gin 1, member 0 [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1450b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2890 (RPC#1137) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:39.573 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2890, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 16:05:39.574 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:05:39.574 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1137 [ CSSD]2013-05-23 16:05:39.580 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:39.580 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:39.580 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:39.580 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1138 (0x7f431bfe2910) [ CSSD]2013-05-23 16:05:39.580 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2910 (RPC#1138) tag(472002c) sent to node 1 [ CSSD]2013-05-23 16:05:39.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0472002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:39.581 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:39.581 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2910 (RPC#1138) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:39.581 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2910, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 16:05:39.581 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1138 [ CSSD]2013-05-23 16:05:39.581 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:05:39.783 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:05:39.783 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 31511 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:40.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:40.955 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:40.977 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:41.832 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:41.832 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:42.961 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:42.961 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:42.989 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:45.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:45.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:45.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:46.835 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:46.836 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:47.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:47.019 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:47.040 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:48.568 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:05:48.569 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1139 (0x7f431bfe2990) [ CSSD]2013-05-23 16:05:48.569 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2990 (RPC#1139) tag(473002c) sent to node 1 [ CSSD]2013-05-23 16:05:48.569 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:48.569 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1139 (0x7f431bfe2990) [ CSSD]2013-05-23 16:05:48.569 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2990 (RPC#1139) state 6, flags 0x202 [ CSSD]2013-05-23 16:05:48.569 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2990, ret 0, client (nil) [ CSSD]2013-05-23 16:05:48.570 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1139 [ CSSD]2013-05-23 16:05:49.031 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:49.031 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:49.062 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:50.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:50.163 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014342, gin 1, member 0 [ CSSD]2013-05-23 16:05:50.163 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:05:50.163 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:05:50.203 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:50.203 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:05:50.203 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:05:51.037 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:51.037 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:51.068 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:51.839 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:51.839 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:53.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:53.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:53.080 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:54.815 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:05:54.816 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:05:54.816 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:05:54.816 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(31664) version 11:1:1:4 [ CSSD]2013-05-23 16:05:54.822 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:05:54.822 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:54.822 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c150b50, requested num -1 [ CSSD]2013-05-23 16:05:54.822 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:54.822 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1140 (0x7f431bfe2a10) [ CSSD]2013-05-23 16:05:54.822 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2a10 (RPC#1140) tag(474002c) sent to node 1 [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0474002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014343, gin 1, member 0 [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1450b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2a10 (RPC#1140) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2a10, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 16:05:54.822 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:05:54.823 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1140 [ CSSD]2013-05-23 16:05:54.828 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:05:54.828 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:54.828 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:05:54.828 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1141 (0x7f431bfe2a90) [ CSSD]2013-05-23 16:05:54.828 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2a90 (RPC#1141) tag(475002c) sent to node 1 [ CSSD]2013-05-23 16:05:54.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0475002c) incarnation(174631051) [ CSSD]2013-05-23 16:05:54.828 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:05:54.828 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2a90 (RPC#1141) state 6, flags 0x100 [ CSSD]2013-05-23 16:05:54.828 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2a90, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 16:05:54.829 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1141 [ CSSD]2013-05-23 16:05:54.829 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:05:55.031 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:05:55.031 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 31664 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:05:55.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:55.063 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:55.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:56.842 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:05:56.842 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:05:57.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:57.089 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:57.111 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:05:58.598 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:05:58.598 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1142 (0x7f431bfe2b10) [ CSSD]2013-05-23 16:05:58.598 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2b10 (RPC#1142) tag(476002c) sent to node 1 [ CSSD]2013-05-23 16:05:58.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:58.599 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1142 (0x7f431bfe2b10) [ CSSD]2013-05-23 16:05:58.599 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2b10 (RPC#1142) state 6, flags 0x202 [ CSSD]2013-05-23 16:05:58.599 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2b10, ret 0, client (nil) [ CSSD]2013-05-23 16:05:58.599 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1142 [ CSSD]2013-05-23 16:05:59.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:05:59.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:05:59.122 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:01.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:01.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:01.148 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:01.845 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:01.846 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:03.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:03.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:03.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:05.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:05.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:05.206 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:05.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:05.415 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014344, gin 1, member 0 [ CSSD]2013-05-23 16:06:05.415 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:06:05.415 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:06:05.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:05.422 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:06:05.422 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:06:06.849 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:06.849 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:07.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:07.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:07.212 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:08.629 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:06:08.629 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1143 (0x7f431bfe2b90) [ CSSD]2013-05-23 16:06:08.629 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2b90 (RPC#1143) tag(477002c) sent to node 1 [ CSSD]2013-05-23 16:06:08.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:08.630 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1143 (0x7f431bfe2b90) [ CSSD]2013-05-23 16:06:08.630 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2b90 (RPC#1143) state 6, flags 0x202 [ CSSD]2013-05-23 16:06:08.630 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2b90, ret 0, client (nil) [ CSSD]2013-05-23 16:06:08.630 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1143 [ CSSD]2013-05-23 16:06:09.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:09.172 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:09.223 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:10.064 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:06:10.065 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:06:10.065 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:06:10.065 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c108370) pid(31814) version 11:1:1:4 [ CSSD]2013-05-23 16:06:10.071 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:06:10.071 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:06:10.071 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 16:06:10.071 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:10.071 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1144 (0x7f431bfe2c10) [ CSSD]2013-05-23 16:06:10.071 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2c10 (RPC#1144) tag(478002c) sent to node 1 [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0478002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014345, gin 1, member 0 [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1450b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2c10 (RPC#1144) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2c10, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:06:10.072 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1144 [ CSSD]2013-05-23 16:06:10.078 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:06:10.078 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:10.078 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:10.078 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1145 (0x7f431bfe2c90) [ CSSD]2013-05-23 16:06:10.078 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2c90 (RPC#1145) tag(479002c) sent to node 1 [ CSSD]2013-05-23 16:06:10.079 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0479002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:10.079 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:10.079 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2c90 (RPC#1145) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:10.079 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2c90, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 16:06:10.079 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1145 [ CSSD]2013-05-23 16:06:10.079 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:06:10.282 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:06:10.282 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0840c0) skgpid ospid 31814 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:11.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:11.178 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:11.248 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:11.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:11.635 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014346, gin 1, member 0 [ CSSD]2013-05-23 16:06:11.635 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:06:11.635 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:06:11.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:11.642 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:06:11.642 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:06:11.852 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:11.852 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:13.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:13.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:13.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:15.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:15.253 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:15.287 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:15.433 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:06:15.434 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:06:15.435 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:06:15.435 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c108370) pid(31892) version 11:1:1:4 [ CSSD]2013-05-23 16:06:15.441 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:06:15.441 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:06:15.441 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c150b50, requested num -1 [ CSSD]2013-05-23 16:06:15.441 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:15.441 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1146 (0x7f431bfe2d10) [ CSSD]2013-05-23 16:06:15.441 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2d10 (RPC#1146) tag(47a002c) sent to node 1 [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(047a002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014347, gin 1, member 0 [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1450b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2d10 (RPC#1146) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2d10, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:06:15.442 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1146 [ CSSD]2013-05-23 16:06:15.448 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:06:15.448 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:15.448 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:15.448 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1147 (0x7f431bfe2d90) [ CSSD]2013-05-23 16:06:15.448 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2d90 (RPC#1147) tag(47b002c) sent to node 1 [ CSSD]2013-05-23 16:06:15.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(047b002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:15.449 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:15.449 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2d90 (RPC#1147) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:15.449 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2d90, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 16:06:15.449 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1147 [ CSSD]2013-05-23 16:06:15.449 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:06:15.651 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:06:15.651 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0840c0) skgpid ospid 31892 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:16.282 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:06:16.283 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:06:16.283 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:06:16.284 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(31901) version 11:1:1:4 [ CSSD]2013-05-23 16:06:16.301 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:06:16.301 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:06:16.301 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0cf720 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 16:06:16.301 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:06:16.301 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1148 (0x7f431bfe2e10) [ CSSD]2013-05-23 16:06:16.301 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2e10 (RPC#1148) tag(47c002c) sent to node 1 [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(047c002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014348, gin 1, member 0 [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1450b0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2e10 (RPC#1148) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2e10, ret 0, client 0x7f431c0cf720 member 0x7f431c11fd30 [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:06:16.302 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1148 [ CSSD]2013-05-23 16:06:16.308 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:06:16.308 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:06:16.308 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:06:16.308 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1149 (0x7f431bfe2e90) [ CSSD]2013-05-23 16:06:16.308 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2e90 (RPC#1149) tag(47d002c) sent to node 1 [ CSSD]2013-05-23 16:06:16.309 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(047d002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:16.309 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:06:16.309 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2e90 (RPC#1149) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:16.309 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2e90, ret 0, client 0x7f431c0cf720 member 0x7f431c11fd30 [ CSSD]2013-05-23 16:06:16.309 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1149 [ CSSD]2013-05-23 16:06:16.309 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:06:16.419 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147170) allocated [ CSSD]2013-05-23 16:06:16.420 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:06:16.421 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147170 - 1,2,3 [ CSSD]2013-05-23 16:06:16.421 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c150b50) proc(0x7f431c147170) pid(31912) version 11:1:1:4 [ CSSD]2013-05-23 16:06:16.446 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:06:16.448 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:06:16.451 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147170 [ CSSD]2013-05-23 16:06:16.451 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147170) con(0x7f431c150b50) skgpid ospid 31912 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:16.612 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:06:16.612 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 31901 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:16.856 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:16.856 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:17.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:17.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:17.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:18.659 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:06:18.659 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1150 (0x7f431bfe2f10) [ CSSD]2013-05-23 16:06:18.659 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2f10 (RPC#1150) tag(47e002c) sent to node 1 [ CSSD]2013-05-23 16:06:18.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:18.660 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1150 (0x7f431bfe2f10) [ CSSD]2013-05-23 16:06:18.660 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2f10 (RPC#1150) state 6, flags 0x202 [ CSSD]2013-05-23 16:06:18.660 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2f10, ret 0, client (nil) [ CSSD]2013-05-23 16:06:18.660 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1150 [ CSSD]2013-05-23 16:06:19.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:19.271 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:19.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:20.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:20.665 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014349, gin 1, member 0 [ CSSD]2013-05-23 16:06:20.665 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:06:20.665 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:06:20.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:20.672 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:06:20.672 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:06:21.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:21.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:21.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:21.859 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:21.860 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:23.219 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:06:23.219 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:06:23.219 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:06:23.219 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:06:23.220 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:06:23.220 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c109f70) skgpid 26646 ospid 26646 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:23.283 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:23.283 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:23.382 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:25.313 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:06:25.314 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:06:25.315 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:06:25.315 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c129c20) pid(31992) version 11:1:1:4 [ CSSD]2013-05-23 16:06:25.321 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c129c20), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:06:25.321 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:06:25.321 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 16:06:25.321 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:25.321 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1151 (0x7f431bfe2f90) [ CSSD]2013-05-23 16:06:25.321 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2f90 (RPC#1151) tag(47f002c) sent to node 1 [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(047f002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014350, gin 1, member 0 [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2f90 (RPC#1151) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2f90, ret 0, client 0x7f431c135120 member 0x7f431c11fd30 [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:06:25.322 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1151 [ CSSD]2013-05-23 16:06:25.328 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:06:25.328 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:25.328 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:25.328 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1152 (0x7f431bfe3010) [ CSSD]2013-05-23 16:06:25.328 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3010 (RPC#1152) tag(480002c) sent to node 1 [ CSSD]2013-05-23 16:06:25.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0480002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:25.329 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:25.329 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3010 (RPC#1152) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:25.329 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3010, ret 0, client 0x7f431c135120 member 0x7f431c11fd30 [ CSSD]2013-05-23 16:06:25.329 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1152 [ CSSD]2013-05-23 16:06:25.329 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:06:25.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:25.369 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:25.408 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:25.531 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:06:25.531 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0840c0) skgpid ospid 31992 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:26.863 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:26.863 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:27.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:27.341 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:27.480 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:28.679 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:06:28.679 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1153 (0x7f431bfe3090) [ CSSD]2013-05-23 16:06:28.679 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3090 (RPC#1153) tag(481002c) sent to node 1 [ CSSD]2013-05-23 16:06:28.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:28.680 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1153 (0x7f431bfe3090) [ CSSD]2013-05-23 16:06:28.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3090 (RPC#1153) state 6, flags 0x202 [ CSSD]2013-05-23 16:06:28.680 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3090, ret 0, client (nil) [ CSSD]2013-05-23 16:06:28.680 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1153 [ CSSD]2013-05-23 16:06:29.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:29.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:29.485 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:30.865 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:30.866 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:06:31.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:31.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:31.510 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:33.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:33.365 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:33.523 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:35.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:35.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:35.549 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:35.869 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:35.869 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:35.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:35.916 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014351, gin 1, member 0 [ CSSD]2013-05-23 16:06:35.916 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:06:35.916 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:06:35.924 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:35.924 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:06:35.924 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:06:37.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:37.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:37.580 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:38.699 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:06:38.699 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1154 (0x7f431bfe3110) [ CSSD]2013-05-23 16:06:38.700 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3110 (RPC#1154) tag(482002c) sent to node 1 [ CSSD]2013-05-23 16:06:38.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:38.700 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1154 (0x7f431bfe3110) [ CSSD]2013-05-23 16:06:38.700 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3110 (RPC#1154) state 6, flags 0x202 [ CSSD]2013-05-23 16:06:38.700 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3110, ret 0, client (nil) [ CSSD]2013-05-23 16:06:38.700 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1154 [ CSSD]2013-05-23 16:06:39.433 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:39.433 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:39.585 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:40.573 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 16:06:40.574 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:06:40.575 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 16:06:40.575 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c150b50) proc(0x7f431c1355f0) pid(32155) version 11:1:1:4 [ CSSD]2013-05-23 16:06:40.581 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1355f0), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:06:40.581 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:06:40.581 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:06:40.581 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:40.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1155 (0x7f431bfe3190) [ CSSD]2013-05-23 16:06:40.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3190 (RPC#1155) tag(483002c) sent to node 1 [ CSSD]2013-05-23 16:06:40.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0483002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:40.581 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014352, gin 1, member 0 [ CSSD]2013-05-23 16:06:40.581 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:06:40.582 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:06:40.582 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:06:40.582 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3190 (RPC#1155) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:40.582 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3190, ret 0, client 0x7f431c10a560 member 0x7f431c13f950 [ CSSD]2013-05-23 16:06:40.582 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:06:40.582 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1155 [ CSSD]2013-05-23 16:06:40.588 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:06:40.588 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:40.588 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:40.588 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1156 (0x7f431bfe3210) [ CSSD]2013-05-23 16:06:40.588 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3210 (RPC#1156) tag(484002c) sent to node 1 [ CSSD]2013-05-23 16:06:40.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0484002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:40.589 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:40.589 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3210 (RPC#1156) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:40.589 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3210, ret 0, client 0x7f431c10a560 member 0x7f431c13f950 [ CSSD]2013-05-23 16:06:40.589 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1156 [ CSSD]2013-05-23 16:06:40.589 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:06:40.791 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 16:06:40.792 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c150b50) skgpid ospid 32155 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:40.872 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:40.872 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:41.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:41.439 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:41.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:43.464 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:43.464 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:43.622 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:45.476 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:45.476 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:45.628 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:45.876 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:45.876 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:47.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:47.482 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:47.681 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:48.720 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:06:48.721 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1157 (0x7f431bfe3290) [ CSSD]2013-05-23 16:06:48.721 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3290 (RPC#1157) tag(485002c) sent to node 1 [ CSSD]2013-05-23 16:06:48.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:48.721 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1157 (0x7f431bfe3290) [ CSSD]2013-05-23 16:06:48.721 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3290 (RPC#1157) state 6, flags 0x202 [ CSSD]2013-05-23 16:06:48.721 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3290, ret 0, client (nil) [ CSSD]2013-05-23 16:06:48.721 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1157 [ CSSD]2013-05-23 16:06:49.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:49.534 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:49.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:49.878 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:49.879 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:06:51.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:51.168 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014353, gin 1, member 0 [ CSSD]2013-05-23 16:06:51.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:06:51.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:06:51.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:51.174 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:06:51.174 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:06:51.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:51.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:51.711 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:53.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:53.565 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:53.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:54.882 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:54.882 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:06:55.214 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:06:55.214 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:06:55.215 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:06:55.215 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22d7cd0) pid(31288/31288) version 11:1:1:4 [ CSSD]2013-05-23 16:06:55.220 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22d7cd0), client(1/0x22dacd0) [ CSSD]2013-05-23 16:06:55.220 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22dacd0) [ CSSD]2013-05-23 16:06:55.220 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:06:55.221 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:06:55.293 [1191196992] >TRACE: clssgmAllocProc: (0x22fc670) allocated [ CSSD]2013-05-23 16:06:55.294 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:06:55.294 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc670 - 1,2,3 [ CSSD]2013-05-23 16:06:55.294 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2303710) proc(0x22fc670) pid(31286/31286) version 11:1:1:4 [ CSSD]2013-05-23 16:06:55.310 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22fc670), client(1/0x22eee80) [ CSSD]2013-05-23 16:06:55.310 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22eee80) [ CSSD]2013-05-23 16:06:55.310 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:06:55.310 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 16:06:55.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:55.575 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:55.730 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:55.823 [1191196992] >TRACE: clssgmAllocProc: (0x205e390) allocated [ CSSD]2013-05-23 16:06:55.824 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:06:55.825 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e390 - 1,2,3 [ CSSD]2013-05-23 16:06:55.825 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dfee0) proc(0x205e390) pid(32331) version 11:1:1:4 [ CSSD]2013-05-23 16:06:55.842 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x205e390), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 16:06:55.842 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:06:55.842 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 16:06:55.842 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:55.842 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1158 (0x7f431bfe3310) [ CSSD]2013-05-23 16:06:55.842 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3310 (RPC#1158) tag(486002c) sent to node 1 [ CSSD]2013-05-23 16:06:55.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0486002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:55.842 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014354, gin 1, member 0 [ CSSD]2013-05-23 16:06:55.842 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:06:55.843 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:06:55.843 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:06:55.843 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3310 (RPC#1158) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:55.843 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3310, ret 0, client 0x7f431c12e6d0 member 0x7f431c135290 [ CSSD]2013-05-23 16:06:55.843 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 16:06:55.843 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1158 [ CSSD]2013-05-23 16:06:55.849 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:06:55.849 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:55.849 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135290) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:06:55.849 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1159 (0x7f431bfe3390) [ CSSD]2013-05-23 16:06:55.849 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3390 (RPC#1159) tag(487002c) sent to node 1 [ CSSD]2013-05-23 16:06:55.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0487002c) incarnation(174631051) [ CSSD]2013-05-23 16:06:55.850 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:06:55.850 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3390 (RPC#1159) state 6, flags 0x100 [ CSSD]2013-05-23 16:06:55.850 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3390, ret 0, client 0x7f431c12e6d0 member 0x7f431c135290 [ CSSD]2013-05-23 16:06:55.850 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1159 [ CSSD]2013-05-23 16:06:55.850 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135290) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:06:56.052 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e390 [ CSSD]2013-05-23 16:06:56.052 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e390) con(0x22dfee0) skgpid ospid 32331 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:06:57.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:57.581 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:57.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:58.443 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 16:06:58.444 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:06:58.444 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 16:06:58.444 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c1355f0) pid(31983/31983) version 11:1:1:4 [ CSSD]2013-05-23 16:06:58.450 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1355f0), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:06:58.450 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:06:58.450 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:06:58.450 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 35 [ CSSD]2013-05-23 16:06:58.740 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:06:58.740 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1160 (0x7f431bfe3410) [ CSSD]2013-05-23 16:06:58.741 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3410 (RPC#1160) tag(488002c) sent to node 1 [ CSSD]2013-05-23 16:06:58.741 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:58.741 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1160 (0x7f431bfe3410) [ CSSD]2013-05-23 16:06:58.741 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3410 (RPC#1160) state 6, flags 0x202 [ CSSD]2013-05-23 16:06:58.741 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3410, ret 0, client (nil) [ CSSD]2013-05-23 16:06:58.741 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1160 [ CSSD]2013-05-23 16:06:59.634 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:06:59.634 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:06:59.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:06:59.886 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:06:59.886 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:01.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:01.640 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:01.812 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:02.505 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1355f0), client(2/0x7f431c12e6d0) [ CSSD]2013-05-23 16:07:02.506 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:07:02.506 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:07:02.506 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:07:03.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:03.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:03.823 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:04.889 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:04.889 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:05.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:05.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:05.848 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:06.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:06.439 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014355, gin 1, member 0 [ CSSD]2013-05-23 16:07:06.439 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:06.439 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:06.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:06.447 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:06.447 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:07.662 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:07.662 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:07.851 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:08.770 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:07:08.771 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1161 (0x7f431bfe3490) [ CSSD]2013-05-23 16:07:08.771 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3490 (RPC#1161) tag(489002c) sent to node 1 [ CSSD]2013-05-23 16:07:08.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:08.771 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1161 (0x7f431bfe3490) [ CSSD]2013-05-23 16:07:08.771 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3490 (RPC#1161) state 6, flags 0x202 [ CSSD]2013-05-23 16:07:08.771 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3490, ret 0, client (nil) [ CSSD]2013-05-23 16:07:08.772 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1161 [ CSSD]2013-05-23 16:07:08.891 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:08.892 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:07:09.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:09.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:09.876 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:11.084 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:07:11.085 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:07:11.085 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:07:11.085 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa10) proc(0x7f431c108370) pid(32474) version 11:1:1:4 [ CSSD]2013-05-23 16:07:11.091 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:07:11.091 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:07:11.091 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c1246d0, requested num -1 [ CSSD]2013-05-23 16:07:11.091 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:11.091 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1162 (0x7f431bfe3510) [ CSSD]2013-05-23 16:07:11.092 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3510 (RPC#1162) tag(48a002c) sent to node 1 [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(048a002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014356, gin 1, member 0 [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c145460/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3510 (RPC#1162) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3510, ret 0, client 0x7f431c0cf720 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:11.092 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:07:11.093 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1162 [ CSSD]2013-05-23 16:07:11.099 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:07:11.099 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:11.099 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:11.099 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1163 (0x7f431bfe3590) [ CSSD]2013-05-23 16:07:11.099 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3590 (RPC#1163) tag(48b002c) sent to node 1 [ CSSD]2013-05-23 16:07:11.099 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(048b002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.099 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:11.099 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3590 (RPC#1163) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:11.099 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3590, ret 0, client 0x7f431c0cf720 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:11.100 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1163 [ CSSD]2013-05-23 16:07:11.100 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:07:11.302 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:07:11.302 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c13fa10) skgpid ospid 32474 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:11.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.530 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014357, gin 1, member 0 [ CSSD]2013-05-23 16:07:11.530 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:11.530 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:11.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.536 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:11.536 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:11.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.680 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:11.881 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:11.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.896 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014358, gin 1, member 0 [ CSSD]2013-05-23 16:07:11.896 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:11.896 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:11.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:11.904 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:11.904 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eed40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:13.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:13.682 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:13.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:13.895 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:13.895 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:14.713 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:07:14.715 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:07:14.715 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:07:14.715 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c10a250) proc(0x7f431c108370) pid(32720) version 11:1:1:4 [ CSSD]2013-05-23 16:07:14.846 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147680) allocated [ CSSD]2013-05-23 16:07:14.848 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:07:14.848 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147680 - 1,2,3 [ CSSD]2013-05-23 16:07:14.848 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c124860) proc(0x7f431c147680) pid(32731) version 11:1:1:4 [ CSSD]2013-05-23 16:07:14.875 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:07:14.876 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:07:14.878 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147680 [ CSSD]2013-05-23 16:07:14.878 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147680) con(0x7f431c124860) skgpid ospid 32731 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:15.042 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(64/0x7f431c0cf720) [ CSSD]2013-05-23 16:07:15.042 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 64 (0x7f431c0cf720) [ CSSD]2013-05-23 16:07:15.042 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c0cf720 with con 0x7f431c1246d0, requested num -1 [ CSSD]2013-05-23 16:07:15.042 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:07:15.042 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1164 (0x7f431bfe3610) [ CSSD]2013-05-23 16:07:15.042 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3610 (RPC#1164) tag(48c002c) sent to node 1 [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(048c002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014359, gin 1, member 0 [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c145460/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3610 (RPC#1164) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3610, ret 0, client 0x7f431c0cf720 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:07:15.043 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1164 [ CSSD]2013-05-23 16:07:15.049 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 64 (0x7f431c0cf720) [ CSSD]2013-05-23 16:07:15.049 [1191196992] >TRACE: clssgmExitGrock: client 64 (0x7f431c0cf720), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:07:15.049 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:07:15.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1165 (0x7f431bfe3690) [ CSSD]2013-05-23 16:07:15.049 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3690 (RPC#1165) tag(48d002c) sent to node 1 [ CSSD]2013-05-23 16:07:15.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(048d002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:15.049 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:07:15.049 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3690 (RPC#1165) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:15.049 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3690, ret 0, client 0x7f431c0cf720 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:15.050 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1165 [ CSSD]2013-05-23 16:07:15.050 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:07:15.050 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:07:15.051 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c10a250) skgpid ospid 32720 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:15.691 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:07:15.692 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:07:15.693 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:07:15.693 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c108370) pid(32741) version 11:1:1:4 [ CSSD]2013-05-23 16:07:15.698 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:07:15.699 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:07:15.699 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c13ea10, requested num -1 [ CSSD]2013-05-23 16:07:15.699 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:15.699 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1166 (0x7f431bfe3710) [ CSSD]2013-05-23 16:07:15.699 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3710 (RPC#1166) tag(48e002c) sent to node 1 [ CSSD]2013-05-23 16:07:15.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(048e002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:15.699 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014360, gin 1, member 0 [ CSSD]2013-05-23 16:07:15.699 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1452d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:07:15.700 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:07:15.700 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:07:15.700 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3710 (RPC#1166) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:15.700 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3710, ret 0, client 0x7f431c10a560 member 0x7f431c1455d0 [ CSSD]2013-05-23 16:07:15.700 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:07:15.700 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1166 [ CSSD]2013-05-23 16:07:15.706 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:07:15.706 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:15.706 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1455d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:15.706 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1167 (0x7f431bfe3790) [ CSSD]2013-05-23 16:07:15.706 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3790 (RPC#1167) tag(48f002c) sent to node 1 [ CSSD]2013-05-23 16:07:15.706 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(048f002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:15.706 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:15.706 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3790 (RPC#1167) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:15.706 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3790, ret 0, client 0x7f431c10a560 member 0x7f431c1455d0 [ CSSD]2013-05-23 16:07:15.707 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1167 [ CSSD]2013-05-23 16:07:15.707 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1455d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:07:15.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:15.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:15.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:15.909 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:07:15.909 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0840c0) skgpid ospid 32741 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:17.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:17.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:17.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:18.793 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:07:18.793 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1168 (0x7f431bfe3810) [ CSSD]2013-05-23 16:07:18.793 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3810 (RPC#1168) tag(490002c) sent to node 1 [ CSSD]2013-05-23 16:07:18.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:18.794 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1168 (0x7f431bfe3810) [ CSSD]2013-05-23 16:07:18.794 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3810 (RPC#1168) state 6, flags 0x202 [ CSSD]2013-05-23 16:07:18.794 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3810, ret 0, client (nil) [ CSSD]2013-05-23 16:07:18.794 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1168 [ CSSD]2013-05-23 16:07:18.898 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:18.898 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:19.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:19.784 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:19.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:21.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:21.708 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014361, gin 1, member 0 [ CSSD]2013-05-23 16:07:21.708 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:21.708 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:21.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:21.715 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:21.715 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:21.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:21.789 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:21.942 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:23.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:23.843 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:23.902 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:23.902 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:23.995 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:25.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:25.868 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:26.021 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:26.341 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:07:26.342 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:07:26.343 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:07:26.343 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11fdf0) proc(0x7f431c108370) pid(353) version 11:1:1:4 [ CSSD]2013-05-23 16:07:26.349 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:07:26.349 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:07:26.349 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c1246d0, requested num -1 [ CSSD]2013-05-23 16:07:26.349 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:26.349 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1169 (0x7f431bfe3890) [ CSSD]2013-05-23 16:07:26.349 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3890 (RPC#1169) tag(491002c) sent to node 1 [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0491002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014362, gin 1, member 0 [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c145460/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3890 (RPC#1169) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:26.349 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3890, ret 0, client 0x7f431c109f80 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:26.350 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:07:26.350 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1169 [ CSSD]2013-05-23 16:07:26.356 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:07:26.356 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:26.356 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:26.356 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1170 (0x7f431bfe3910) [ CSSD]2013-05-23 16:07:26.356 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3910 (RPC#1170) tag(492002c) sent to node 1 [ CSSD]2013-05-23 16:07:26.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0492002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:26.356 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:26.356 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3910 (RPC#1170) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:26.356 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3910, ret 0, client 0x7f431c109f80 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:26.357 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1170 [ CSSD]2013-05-23 16:07:26.357 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:07:26.559 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:07:26.559 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11fdf0) skgpid ospid 353 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:27.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:27.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:27.905 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:27.905 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:07:28.111 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:28.818 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:07:28.818 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1171 (0x7f431bfe3990) [ CSSD]2013-05-23 16:07:28.818 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3990 (RPC#1171) tag(493002c) sent to node 1 [ CSSD]2013-05-23 16:07:28.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:28.819 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1171 (0x7f431bfe3990) [ CSSD]2013-05-23 16:07:28.819 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3990 (RPC#1171) state 6, flags 0x202 [ CSSD]2013-05-23 16:07:28.819 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3990, ret 0, client (nil) [ CSSD]2013-05-23 16:07:28.819 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1171 [ CSSD]2013-05-23 16:07:29.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:29.884 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:30.136 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:31.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:31.890 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:32.142 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:32.908 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:32.908 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:33.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:33.943 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:34.194 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:35.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:35.949 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:36.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:36.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:36.958 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014363, gin 1, member 0 [ CSSD]2013-05-23 16:07:36.958 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:36.958 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:36.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:36.965 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:36.965 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302b40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:37.911 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:37.911 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:37.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:37.974 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:38.232 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:38.841 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:07:38.841 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1172 (0x7f431bfe3a10) [ CSSD]2013-05-23 16:07:38.841 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3a10 (RPC#1172) tag(494002c) sent to node 1 [ CSSD]2013-05-23 16:07:38.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:38.842 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1172 (0x7f431bfe3a10) [ CSSD]2013-05-23 16:07:38.842 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3a10 (RPC#1172) state 6, flags 0x202 [ CSSD]2013-05-23 16:07:38.842 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3a10, ret 0, client (nil) [ CSSD]2013-05-23 16:07:38.842 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1172 [ CSSD]2013-05-23 16:07:39.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:39.985 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:40.238 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:41.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:41.060 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014364, gin 1, member 0 [ CSSD]2013-05-23 16:07:41.060 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:41.060 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:41.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:41.066 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:41.067 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302b40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:41.591 [1191196992] >TRACE: clssgmAllocProc: (0x205e390) allocated [ CSSD]2013-05-23 16:07:41.592 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:07:41.593 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e390 - 1,2,3 [ CSSD]2013-05-23 16:07:41.593 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dfee0) proc(0x205e390) pid(494) version 11:1:1:4 [ CSSD]2013-05-23 16:07:41.598 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x205e390), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:07:41.598 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:07:41.598 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:07:41.598 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:41.598 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1173 (0x7f431bfe3a90) [ CSSD]2013-05-23 16:07:41.599 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3a90 (RPC#1173) tag(495002c) sent to node 1 [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0495002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014365, gin 1, member 0 [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3a90 (RPC#1173) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3a90, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 16:07:41.599 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:07:41.600 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1173 [ CSSD]2013-05-23 16:07:41.606 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:07:41.606 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:41.606 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:41.606 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1174 (0x7f431bfe3b10) [ CSSD]2013-05-23 16:07:41.606 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3b10 (RPC#1174) tag(496002c) sent to node 1 [ CSSD]2013-05-23 16:07:41.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0496002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:41.607 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:41.607 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3b10 (RPC#1174) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:41.607 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3b10, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 16:07:41.607 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1174 [ CSSD]2013-05-23 16:07:41.607 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:07:41.810 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e390 [ CSSD]2013-05-23 16:07:41.810 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e390) con(0x22dfee0) skgpid ospid 494 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:41.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:41.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:42.291 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:42.914 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:42.914 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:44.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:44.044 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:44.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:46.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:46.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:46.323 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:47.917 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:47.917 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:48.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:48.056 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:48.333 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:48.864 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:07:48.864 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1175 (0x7f431bfe3b90) [ CSSD]2013-05-23 16:07:48.864 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3b90 (RPC#1175) tag(497002c) sent to node 1 [ CSSD]2013-05-23 16:07:48.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:48.865 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1175 (0x7f431bfe3b90) [ CSSD]2013-05-23 16:07:48.865 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3b90 (RPC#1175) state 6, flags 0x202 [ CSSD]2013-05-23 16:07:48.865 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3b90, ret 0, client (nil) [ CSSD]2013-05-23 16:07:48.865 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1175 [ CSSD]2013-05-23 16:07:50.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:50.066 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:50.339 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:52.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:52.092 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:52.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:52.210 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014366, gin 1, member 0 [ CSSD]2013-05-23 16:07:52.210 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:07:52.210 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:07:52.217 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:52.217 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:07:52.217 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:07:52.391 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:52.920 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:52.920 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:54.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:54.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:54.397 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:56.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:56.149 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:56.403 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:56.842 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:07:56.843 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:07:56.843 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:07:56.843 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa10) proc(0x7f431c108370) pid(723) version 11:1:1:4 [ CSSD]2013-05-23 16:07:56.850 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:07:56.850 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:07:56.850 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c1246d0, requested num -1 [ CSSD]2013-05-23 16:07:56.850 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:56.850 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1176 (0x7f431bfe3c10) [ CSSD]2013-05-23 16:07:56.850 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3c10 (RPC#1176) tag(498002c) sent to node 1 [ CSSD]2013-05-23 16:07:56.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0498002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:56.850 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014367, gin 1, member 0 [ CSSD]2013-05-23 16:07:56.850 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c145460/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:07:56.851 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:07:56.851 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:07:56.851 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3c10 (RPC#1176) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:56.851 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3c10, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:56.851 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:07:56.851 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1176 [ CSSD]2013-05-23 16:07:56.856 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:07:56.856 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:56.856 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:07:56.856 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1177 (0x7f431bfe3c90) [ CSSD]2013-05-23 16:07:56.856 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3c90 (RPC#1177) tag(499002c) sent to node 1 [ CSSD]2013-05-23 16:07:56.857 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0499002c) incarnation(174631051) [ CSSD]2013-05-23 16:07:56.857 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:07:56.857 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3c90 (RPC#1177) state 6, flags 0x100 [ CSSD]2013-05-23 16:07:56.857 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3c90, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:07:56.857 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1177 [ CSSD]2013-05-23 16:07:56.857 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:07:57.059 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:07:57.059 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c13fa10) skgpid ospid 723 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:07:57.924 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:07:57.924 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:07:58.155 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:58.155 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:07:58.413 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:07:58.888 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:07:58.888 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1178 (0x7f431bfe3d10) [ CSSD]2013-05-23 16:07:58.888 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3d10 (RPC#1178) tag(49a002c) sent to node 1 [ CSSD]2013-05-23 16:07:58.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:07:58.889 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1178 (0x7f431bfe3d10) [ CSSD]2013-05-23 16:07:58.889 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3d10 (RPC#1178) state 6, flags 0x202 [ CSSD]2013-05-23 16:07:58.889 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3d10, ret 0, client (nil) [ CSSD]2013-05-23 16:07:58.889 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1178 [ CSSD]2013-05-23 16:08:00.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:00.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:00.458 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:02.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:02.192 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:02.492 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:02.927 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:02.927 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:04.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:04.244 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:04.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:06.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:06.250 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:06.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:06.930 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:06.930 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:08:07.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:07.461 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014368, gin 1, member 0 [ CSSD]2013-05-23 16:08:07.461 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:08:07.461 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:08:07.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:07.467 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:08:07.467 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:08:08.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:08.256 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:08.514 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:08.912 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:08:08.912 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1179 (0x7f431bfe3d90) [ CSSD]2013-05-23 16:08:08.913 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3d90 (RPC#1179) tag(49b002c) sent to node 1 [ CSSD]2013-05-23 16:08:08.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:08.913 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1179 (0x7f431bfe3d90) [ CSSD]2013-05-23 16:08:08.913 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3d90 (RPC#1179) state 6, flags 0x202 [ CSSD]2013-05-23 16:08:08.913 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3d90, ret 0, client (nil) [ CSSD]2013-05-23 16:08:08.914 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1179 [ CSSD]2013-05-23 16:08:10.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:10.267 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:10.520 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:11.933 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:11.933 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:12.091 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:08:12.092 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:08:12.092 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:08:12.092 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa10) proc(0x7f431c108370) pid(876) version 11:1:1:4 [ CSSD]2013-05-23 16:08:12.098 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:08:12.098 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:12.098 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c1246d0, requested num -1 [ CSSD]2013-05-23 16:08:12.098 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:12.098 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1180 (0x7f431bfe3e10) [ CSSD]2013-05-23 16:08:12.098 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3e10 (RPC#1180) tag(49c002c) sent to node 1 [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(049c002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014369, gin 1, member 0 [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c145460/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3e10 (RPC#1180) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3e10, ret 0, client 0x7f431c109f80 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:08:12.099 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1180 [ CSSD]2013-05-23 16:08:12.105 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:12.105 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:12.105 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:12.105 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1181 (0x7f431bfe3e90) [ CSSD]2013-05-23 16:08:12.105 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3e90 (RPC#1181) tag(49d002c) sent to node 1 [ CSSD]2013-05-23 16:08:12.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(049d002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:12.105 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:12.105 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3e90 (RPC#1181) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:12.105 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3e90, ret 0, client 0x7f431c109f80 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:08:12.106 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1181 [ CSSD]2013-05-23 16:08:12.106 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:08:12.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:12.151 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014370, gin 1, member 0 [ CSSD]2013-05-23 16:08:12.151 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:08:12.151 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:08:12.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:12.158 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:08:12.158 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:08:12.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:12.273 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:12.308 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:08:12.308 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c13fa10) skgpid ospid 876 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:12.592 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:14.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:14.285 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:14.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:15.951 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:08:15.952 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:08:15.952 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:08:15.952 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11fdf0) proc(0x7f431c108370) pid(941) version 11:1:1:4 [ CSSD]2013-05-23 16:08:15.958 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:08:15.958 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:08:15.958 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c1246d0, requested num -1 [ CSSD]2013-05-23 16:08:15.958 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:15.958 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1182 (0x7f431bfe3f10) [ CSSD]2013-05-23 16:08:15.958 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3f10 (RPC#1182) tag(49e002c) sent to node 1 [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(049e002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014371, gin 1, member 0 [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c145460/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3f10 (RPC#1182) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3f10, ret 0, client 0x7f431c0cf720 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:08:15.959 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1182 [ CSSD]2013-05-23 16:08:15.965 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:08:15.965 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:15.965 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:15.965 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1183 (0x7f431bfe3f90) [ CSSD]2013-05-23 16:08:15.965 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3f90 (RPC#1183) tag(49f002c) sent to node 1 [ CSSD]2013-05-23 16:08:15.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(049f002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:15.965 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:15.965 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3f90 (RPC#1183) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:15.965 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3f90, ret 0, client 0x7f431c0cf720 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:08:15.965 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1183 [ CSSD]2013-05-23 16:08:15.965 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:08:16.168 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:08:16.168 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11fdf0) skgpid ospid 941 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:16.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:16.291 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:16.624 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:16.936 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:16.936 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:18.337 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:18.337 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:18.635 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:18.936 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:08:18.937 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1184 (0x7f431bfe4010) [ CSSD]2013-05-23 16:08:18.937 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4010 (RPC#1184) tag(4a0002c) sent to node 1 [ CSSD]2013-05-23 16:08:18.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:18.937 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1184 (0x7f431bfe4010) [ CSSD]2013-05-23 16:08:18.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4010 (RPC#1184) state 6, flags 0x202 [ CSSD]2013-05-23 16:08:18.937 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4010, ret 0, client (nil) [ CSSD]2013-05-23 16:08:18.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1184 [ CSSD]2013-05-23 16:08:20.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:20.348 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:20.662 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:21.939 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:21.940 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:22.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:22.354 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:22.692 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:22.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:22.712 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014372, gin 1, member 0 [ CSSD]2013-05-23 16:08:22.712 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:08:22.712 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:08:22.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:22.719 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:08:22.719 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:08:24.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:24.365 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:24.697 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:25.905 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:08:25.906 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:08:25.906 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:08:25.906 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa10) proc(0x7f431c108370) pid(647/647) version 11:1:1:4 [ CSSD]2013-05-23 16:08:25.911 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c108370), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:08:25.911 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:08:25.911 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:08:25.911 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 36 [ CSSD]2013-05-23 16:08:26.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:26.391 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:26.703 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:26.942 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:26.943 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:27.351 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147ab0) allocated [ CSSD]2013-05-23 16:08:27.352 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:08:27.353 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147ab0 - 1,2,3 [ CSSD]2013-05-23 16:08:27.353 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c145810) proc(0x7f431c147ab0) pid(1055) version 11:1:1:4 [ CSSD]2013-05-23 16:08:27.359 [1191196992] >TRACE: clssgmRegisterClient: proc(65/0x7f431c147ab0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:08:27.359 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:27.359 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c147a00, requested num -1 [ CSSD]2013-05-23 16:08:27.359 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:27.359 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1185 (0x7f431bfe4090) [ CSSD]2013-05-23 16:08:27.359 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4090 (RPC#1185) tag(4a1002c) sent to node 1 [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04a1002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014373, gin 1, member 0 [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c148240/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4090 (RPC#1185) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4090, ret 0, client 0x7f431c109f80 member 0x7f431c148540 [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:08:27.360 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1185 [ CSSD]2013-05-23 16:08:27.367 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:27.367 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:27.367 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c148540) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:27.367 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1186 (0x7f431bfe4110) [ CSSD]2013-05-23 16:08:27.367 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4110 (RPC#1186) tag(4a2002c) sent to node 1 [ CSSD]2013-05-23 16:08:27.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04a2002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:27.367 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:27.367 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4110 (RPC#1186) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:27.367 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4110, ret 0, client 0x7f431c109f80 member 0x7f431c148540 [ CSSD]2013-05-23 16:08:27.368 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1186 [ CSSD]2013-05-23 16:08:27.368 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c148540) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:08:27.570 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147ab0 [ CSSD]2013-05-23 16:08:27.570 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147ab0) con(0x7f431c145810) skgpid ospid 1055 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:28.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:28.397 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:28.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:28.960 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:08:28.961 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1187 (0x7f431bfe4190) [ CSSD]2013-05-23 16:08:28.961 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4190 (RPC#1187) tag(4a3002c) sent to node 1 [ CSSD]2013-05-23 16:08:28.961 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:28.961 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1187 (0x7f431bfe4190) [ CSSD]2013-05-23 16:08:28.961 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4190 (RPC#1187) state 6, flags 0x202 [ CSSD]2013-05-23 16:08:28.961 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4190, ret 0, client (nil) [ CSSD]2013-05-23 16:08:28.961 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1187 [ CSSD]2013-05-23 16:08:30.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:30.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:30.731 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:30.945 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:30.945 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:08:32.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:32.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:32.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:34.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:34.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:34.749 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:35.948 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:35.948 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:36.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:36.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:36.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:37.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:37.462 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014374, gin 1, member 0 [ CSSD]2013-05-23 16:08:37.463 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:08:37.463 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:08:37.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:37.469 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:08:37.469 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:08:37.594 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:08:37.595 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:08:37.596 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:08:37.596 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:08:37.597 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:08:37.598 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:08:37.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:37.963 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014375, gin 1, member 0 [ CSSD]2013-05-23 16:08:37.963 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:08:37.963 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:08:37.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:37.970 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:08:37.970 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eed40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:08:38.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:38.496 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:38.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:38.983 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:08:38.984 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1188 (0x7f431bfe4210) [ CSSD]2013-05-23 16:08:38.984 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4210 (RPC#1188) tag(4a4002c) sent to node 1 [ CSSD]2013-05-23 16:08:38.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:38.984 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1188 (0x7f431bfe4210) [ CSSD]2013-05-23 16:08:38.984 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4210 (RPC#1188) state 6, flags 0x202 [ CSSD]2013-05-23 16:08:38.984 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4210, ret 0, client (nil) [ CSSD]2013-05-23 16:08:38.985 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1188 [ CSSD]2013-05-23 16:08:40.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:40.499 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:40.812 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:40.952 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:40.952 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:42.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:42.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:42.614 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1479a0) allocated [ CSSD]2013-05-23 16:08:42.615 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:08:42.616 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1479a0 - 1,2,3 [ CSSD]2013-05-23 16:08:42.616 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c1479a0) pid(1212) version 11:1:1:4 [ CSSD]2013-05-23 16:08:42.632 [1191196992] >TRACE: clssgmRegisterClient: proc(65/0x7f431c1479a0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:08:42.632 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:42.632 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c1478f0, requested num -1 [ CSSD]2013-05-23 16:08:42.632 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:42.632 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1189 (0x7f431bfe4290) [ CSSD]2013-05-23 16:08:42.632 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4290 (RPC#1189) tag(4a5002c) sent to node 1 [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04a5002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014376, gin 1, member 0 [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c148130/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4290 (RPC#1189) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4290, ret 0, client 0x7f431c109f80 member 0x7f431c148430 [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:08:42.633 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1189 [ CSSD]2013-05-23 16:08:42.639 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:42.639 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:42.639 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c148430) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:42.639 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1190 (0x7f431bfe4310) [ CSSD]2013-05-23 16:08:42.639 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4310 (RPC#1190) tag(4a6002c) sent to node 1 [ CSSD]2013-05-23 16:08:42.639 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04a6002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:42.639 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:42.639 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4310 (RPC#1190) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:42.639 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4310, ret 0, client 0x7f431c109f80 member 0x7f431c148430 [ CSSD]2013-05-23 16:08:42.640 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1190 [ CSSD]2013-05-23 16:08:42.640 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c148430) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:08:42.823 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:42.842 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1479a0 [ CSSD]2013-05-23 16:08:42.842 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1479a0) con(0x7f431c0840c0) skgpid ospid 1212 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:44.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:44.556 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:44.830 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:45.956 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:45.956 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:46.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:46.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:46.836 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:48.528 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147ab0) allocated [ CSSD]2013-05-23 16:08:48.529 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:08:48.530 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147ab0 - 1,2,3 [ CSSD]2013-05-23 16:08:48.530 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c145810) proc(0x7f431c147ab0) pid(645/645) version 11:1:1:4 [ CSSD]2013-05-23 16:08:48.545 [1191196992] >TRACE: clssgmRegisterClient: proc(65/0x7f431c147ab0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:08:48.545 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:08:48.545 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:08:48.545 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 37 [ CSSD]2013-05-23 16:08:48.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:48.587 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:48.846 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:49.010 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:08:49.010 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1191 (0x7f431bfe4390) [ CSSD]2013-05-23 16:08:49.011 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4390 (RPC#1191) tag(4a7002c) sent to node 1 [ CSSD]2013-05-23 16:08:49.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:49.011 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1191 (0x7f431bfe4390) [ CSSD]2013-05-23 16:08:49.011 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4390 (RPC#1191) state 6, flags 0x202 [ CSSD]2013-05-23 16:08:49.011 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4390, ret 0, client (nil) [ CSSD]2013-05-23 16:08:49.011 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1191 [ CSSD]2013-05-23 16:08:50.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:50.598 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:50.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:50.959 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:50.960 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:52.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:52.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:52.942 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:53.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:53.222 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014377, gin 1, member 0 [ CSSD]2013-05-23 16:08:53.222 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:08:53.222 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:08:53.226 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:08:53.226 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:08:53.226 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:08:53.226 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 36 [ CSSD]2013-05-23 16:08:53.226 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 16:08:53.226 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c0e6720) skgpid 28479 ospid 28479 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:53.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:53.228 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:08:53.228 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:08:53.265 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:08:53.265 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:08:53.265 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:08:53.266 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 35 [ CSSD]2013-05-23 16:08:53.266 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135f60 [ CSSD]2013-05-23 16:08:53.266 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135f60) con(0x7f431c0e6e20) skgpid 28493 ospid 28493 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:54.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:54.657 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:54.949 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:55.963 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:08:55.963 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:08:56.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:56.663 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:56.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:57.889 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:08:57.890 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:08:57.891 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:08:57.891 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c147f40) pid(1389) version 11:1:1:4 [ CSSD]2013-05-23 16:08:57.897 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:08:57.897 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:08:57.897 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:08:57.897 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:57.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1192 (0x7f431bfe4410) [ CSSD]2013-05-23 16:08:57.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4410 (RPC#1192) tag(4a8002c) sent to node 1 [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04a8002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014378, gin 1, member 0 [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4410 (RPC#1192) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4410, ret 0, client 0x7f431c0e5910 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:08:57.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1192 [ CSSD]2013-05-23 16:08:57.904 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:08:57.904 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:57.904 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:08:57.904 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1193 (0x7f431bfe4490) [ CSSD]2013-05-23 16:08:57.904 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4490 (RPC#1193) tag(4a9002c) sent to node 1 [ CSSD]2013-05-23 16:08:57.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04a9002c) incarnation(174631051) [ CSSD]2013-05-23 16:08:57.905 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:08:57.905 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4490 (RPC#1193) state 6, flags 0x100 [ CSSD]2013-05-23 16:08:57.905 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4490, ret 0, client 0x7f431c0e5910 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:08:57.905 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1193 [ CSSD]2013-05-23 16:08:57.905 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:08:58.108 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:08:58.108 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c11ab50) skgpid ospid 1389 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:08:58.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:58.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:08:59.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:08:59.040 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:08:59.041 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1194 (0x7f431bfe4510) [ CSSD]2013-05-23 16:08:59.041 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4510 (RPC#1194) tag(4aa002c) sent to node 1 [ CSSD]2013-05-23 16:08:59.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:08:59.041 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1194 (0x7f431bfe4510) [ CSSD]2013-05-23 16:08:59.041 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4510 (RPC#1194) state 6, flags 0x202 [ CSSD]2013-05-23 16:08:59.041 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4510, ret 0, client (nil) [ CSSD]2013-05-23 16:08:59.042 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1194 [ CSSD]2013-05-23 16:09:00.780 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:00.780 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:00.967 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:00.967 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:01.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:02.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:02.786 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:03.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:04.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:04.796 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:04.970 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:04.970 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:09:05.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:06.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:06.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:07.055 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:08.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:08.472 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014379, gin 1, member 0 [ CSSD]2013-05-23 16:09:08.472 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:09:08.472 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:09:08.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:08.480 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:09:08.480 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:09:08.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:08.855 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:09.063 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:09:09.063 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1195 (0x7f431bfe4590) [ CSSD]2013-05-23 16:09:09.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4590 (RPC#1195) tag(4ab002c) sent to node 1 [ CSSD]2013-05-23 16:09:09.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:09.064 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1195 (0x7f431bfe4590) [ CSSD]2013-05-23 16:09:09.064 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4590 (RPC#1195) state 6, flags 0x202 [ CSSD]2013-05-23 16:09:09.064 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4590, ret 0, client (nil) [ CSSD]2013-05-23 16:09:09.064 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1195 [ CSSD]2013-05-23 16:09:09.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:09.973 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:09.973 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:10.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:10.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:11.135 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:12.414 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:12.414 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014380, gin 1, member 0 [ CSSD]2013-05-23 16:09:12.415 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:09:12.415 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:09:12.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:12.421 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:09:12.421 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:09:12.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:12.865 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:13.143 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:13.144 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:09:13.145 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:09:13.146 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:09:13.146 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c147f40) pid(1537) version 11:1:1:4 [ CSSD]2013-05-23 16:09:13.153 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:09:13.153 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:09:13.153 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c147a00, requested num -1 [ CSSD]2013-05-23 16:09:13.153 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:13.153 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1196 (0x7f431bfe4610) [ CSSD]2013-05-23 16:09:13.153 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4610 (RPC#1196) tag(4ac002c) sent to node 1 [ CSSD]2013-05-23 16:09:13.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ac002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:13.153 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014381, gin 1, member 0 [ CSSD]2013-05-23 16:09:13.153 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:09:13.154 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:09:13.154 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:09:13.154 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4610 (RPC#1196) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:13.154 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4610, ret 0, client 0x7f431c0cf720 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:13.154 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:09:13.154 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1196 [ CSSD]2013-05-23 16:09:13.160 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:09:13.160 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:13.160 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:13.160 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1197 (0x7f431bfe4690) [ CSSD]2013-05-23 16:09:13.160 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4690 (RPC#1197) tag(4ad002c) sent to node 1 [ CSSD]2013-05-23 16:09:13.160 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04ad002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:13.160 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:13.161 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4690 (RPC#1197) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:13.161 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4690, ret 0, client 0x7f431c0cf720 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:13.161 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1197 [ CSSD]2013-05-23 16:09:13.161 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:09:13.363 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:09:13.363 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c0840c0) skgpid ospid 1537 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:09:14.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:14.876 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:14.977 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:14.977 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:15.151 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:16.219 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:09:16.221 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:09:16.221 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:09:16.221 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c147f40) pid(1596) version 11:1:1:4 [ CSSD]2013-05-23 16:09:16.228 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:09:16.228 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:09:16.228 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:09:16.228 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:16.228 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1198 (0x7f431bfe4710) [ CSSD]2013-05-23 16:09:16.229 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4710 (RPC#1198) tag(4ae002c) sent to node 1 [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ae002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014382, gin 1, member 0 [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4710 (RPC#1198) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4710, ret 0, client 0x7f431c121510 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:09:16.229 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1198 [ CSSD]2013-05-23 16:09:16.237 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:09:16.237 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:16.237 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:16.237 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1199 (0x7f431bfe4790) [ CSSD]2013-05-23 16:09:16.237 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4790 (RPC#1199) tag(4af002c) sent to node 1 [ CSSD]2013-05-23 16:09:16.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04af002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:16.237 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:16.237 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4790 (RPC#1199) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:16.237 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4790, ret 0, client 0x7f431c121510 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:16.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1199 [ CSSD]2013-05-23 16:09:16.237 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:09:16.440 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:09:16.440 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c11ab50) skgpid ospid 1596 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:09:16.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:16.882 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:17.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:18.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:18.955 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:19.090 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:09:19.091 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1200 (0x7f431bfe4810) [ CSSD]2013-05-23 16:09:19.091 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4810 (RPC#1200) tag(4b0002c) sent to node 1 [ CSSD]2013-05-23 16:09:19.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:19.091 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1200 (0x7f431bfe4810) [ CSSD]2013-05-23 16:09:19.091 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4810 (RPC#1200) state 6, flags 0x202 [ CSSD]2013-05-23 16:09:19.091 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4810, ret 0, client (nil) [ CSSD]2013-05-23 16:09:19.092 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1200 [ CSSD]2013-05-23 16:09:19.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:19.981 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:19.981 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:20.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:20.980 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:21.216 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:22.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:22.985 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:23.226 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:23.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:23.723 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014383, gin 1, member 0 [ CSSD]2013-05-23 16:09:23.723 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:09:23.723 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:09:23.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:23.729 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:09:23.729 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:09:24.452 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:09:24.453 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:09:24.454 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:09:24.454 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:09:24.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:24.539 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014384, gin 1, member 0 [ CSSD]2013-05-23 16:09:24.539 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:09:24.539 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:09:24.546 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:24.546 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:09:24.546 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:09:24.985 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:24.985 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:24.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:24.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:25.233 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:27.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:27.003 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:27.239 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:28.393 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:09:28.394 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:09:28.394 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:09:28.394 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c147f40) pid(1684) version 11:1:1:4 [ CSSD]2013-05-23 16:09:28.401 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:09:28.401 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:09:28.401 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c147a00, requested num -1 [ CSSD]2013-05-23 16:09:28.401 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:28.401 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1201 (0x7f431bfe4890) [ CSSD]2013-05-23 16:09:28.401 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4890 (RPC#1201) tag(4b1002c) sent to node 1 [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04b1002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014385, gin 1, member 0 [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4890 (RPC#1201) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4890, ret 0, client 0x7f431c0e5910 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:09:28.402 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1201 [ CSSD]2013-05-23 16:09:28.408 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:09:28.408 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:28.408 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:28.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1202 (0x7f431bfe4910) [ CSSD]2013-05-23 16:09:28.409 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4910 (RPC#1202) tag(4b2002c) sent to node 1 [ CSSD]2013-05-23 16:09:28.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04b2002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:28.409 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:28.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4910 (RPC#1202) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:28.409 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4910, ret 0, client 0x7f431c0e5910 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:28.409 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1202 [ CSSD]2013-05-23 16:09:28.409 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:09:28.612 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:09:28.612 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c0e6e20) skgpid ospid 1684 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:09:29.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:29.056 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:29.115 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:09:29.116 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1203 (0x7f431bfe4990) [ CSSD]2013-05-23 16:09:29.116 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4990 (RPC#1203) tag(4b3002c) sent to node 1 [ CSSD]2013-05-23 16:09:29.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:29.116 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1203 (0x7f431bfe4990) [ CSSD]2013-05-23 16:09:29.116 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4990 (RPC#1203) state 6, flags 0x202 [ CSSD]2013-05-23 16:09:29.116 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4990, ret 0, client (nil) [ CSSD]2013-05-23 16:09:29.116 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1203 [ CSSD]2013-05-23 16:09:29.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:29.988 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:29.989 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:31.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:31.060 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:31.294 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:33.065 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:33.065 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:33.306 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:34.992 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:34.993 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:35.078 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:35.079 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:35.314 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:37.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:37.083 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:37.324 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:38.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:38.989 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014386, gin 1, member 0 [ CSSD]2013-05-23 16:09:38.989 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:09:38.989 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:09:38.996 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:38.996 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:09:38.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:38.996 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:09:38.996 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:09:39.139 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:09:39.140 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1204 (0x7f431bfe4a10) [ CSSD]2013-05-23 16:09:39.140 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4a10 (RPC#1204) tag(4b4002c) sent to node 1 [ CSSD]2013-05-23 16:09:39.141 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:39.141 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1204 (0x7f431bfe4a10) [ CSSD]2013-05-23 16:09:39.141 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4a10 (RPC#1204) state 6, flags 0x202 [ CSSD]2013-05-23 16:09:39.141 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4a10, ret 0, client (nil) [ CSSD]2013-05-23 16:09:39.141 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1204 [ CSSD]2013-05-23 16:09:39.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:39.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:39.350 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:41.180 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:41.181 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:41.357 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:43.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:43.225 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:43.407 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:43.649 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:09:43.650 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:09:43.650 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:09:43.650 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c147f40) pid(1845) version 11:1:1:4 [ CSSD]2013-05-23 16:09:43.656 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:09:43.656 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:09:43.656 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:09:43.656 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:43.656 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1205 (0x7f431bfe4a90) [ CSSD]2013-05-23 16:09:43.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4a90 (RPC#1205) tag(4b5002c) sent to node 1 [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04b5002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014387, gin 1, member 0 [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4a90 (RPC#1205) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4a90, ret 0, client 0x7f431c0cf720 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:43.657 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:09:43.658 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1205 [ CSSD]2013-05-23 16:09:43.664 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:09:43.664 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:43.664 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:43.664 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1206 (0x7f431bfe4b10) [ CSSD]2013-05-23 16:09:43.665 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4b10 (RPC#1206) tag(4b6002c) sent to node 1 [ CSSD]2013-05-23 16:09:43.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04b6002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:43.665 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:43.665 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4b10 (RPC#1206) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:43.665 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4b10, ret 0, client 0x7f431c0cf720 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:43.665 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1206 [ CSSD]2013-05-23 16:09:43.665 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:09:43.868 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:09:43.868 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c12f0c0) skgpid ospid 1845 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:09:43.999 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:44.000 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:45.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:45.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:45.414 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:47.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:47.282 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:47.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:49.003 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:49.003 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:49.166 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:09:49.166 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1207 (0x7f431bfe4b90) [ CSSD]2013-05-23 16:09:49.167 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4b90 (RPC#1207) tag(4b7002c) sent to node 1 [ CSSD]2013-05-23 16:09:49.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:49.167 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1207 (0x7f431bfe4b90) [ CSSD]2013-05-23 16:09:49.167 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4b90 (RPC#1207) state 6, flags 0x202 [ CSSD]2013-05-23 16:09:49.167 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4b90, ret 0, client (nil) [ CSSD]2013-05-23 16:09:49.167 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1207 [ CSSD]2013-05-23 16:09:49.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:49.296 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:49.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:51.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:51.301 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:51.438 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:53.006 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:53.006 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:09:53.305 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:53.306 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:53.457 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:54.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:54.240 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014388, gin 1, member 0 [ CSSD]2013-05-23 16:09:54.240 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:09:54.240 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:09:54.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:54.246 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:09:54.246 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:09:55.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:55.329 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:55.464 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:57.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:57.334 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:57.515 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:09:58.009 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:09:58.010 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:09:58.903 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:09:58.905 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:09:58.905 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:09:58.905 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c147f40) pid(2010) version 11:1:1:4 [ CSSD]2013-05-23 16:09:58.911 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:09:58.911 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:09:58.911 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:09:58.911 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:58.911 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1208 (0x7f431bfe4c10) [ CSSD]2013-05-23 16:09:58.911 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4c10 (RPC#1208) tag(4b8002c) sent to node 1 [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04b8002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014389, gin 1, member 0 [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4c10 (RPC#1208) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4c10, ret 0, client 0x7f431c121510 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:09:58.912 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1208 [ CSSD]2013-05-23 16:09:58.918 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:09:58.918 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:58.918 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:09:58.918 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1209 (0x7f431bfe4c90) [ CSSD]2013-05-23 16:09:58.919 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4c90 (RPC#1209) tag(4b9002c) sent to node 1 [ CSSD]2013-05-23 16:09:58.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04b9002c) incarnation(174631051) [ CSSD]2013-05-23 16:09:58.919 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:09:58.919 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4c90 (RPC#1209) state 6, flags 0x100 [ CSSD]2013-05-23 16:09:58.919 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4c90, ret 0, client 0x7f431c121510 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:09:58.919 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1209 [ CSSD]2013-05-23 16:09:58.919 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:09:59.122 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:09:59.122 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c0e6e20) skgpid ospid 2010 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:09:59.195 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:09:59.196 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1210 (0x7f431bfe4d10) [ CSSD]2013-05-23 16:09:59.196 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4d10 (RPC#1210) tag(4ba002c) sent to node 1 [ CSSD]2013-05-23 16:09:59.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:59.196 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1210 (0x7f431bfe4d10) [ CSSD]2013-05-23 16:09:59.196 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4d10 (RPC#1210) state 6, flags 0x202 [ CSSD]2013-05-23 16:09:59.196 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4d10, ret 0, client (nil) [ CSSD]2013-05-23 16:09:59.196 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1210 [ CSSD]2013-05-23 16:09:59.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:09:59.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:09:59.522 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:01.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:01.372 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:01.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:03.013 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:03.014 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:03.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:03.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:03.537 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:05.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:05.390 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:05.544 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:07.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:07.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:07.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:07.997 [1191196992] >TRACE: clssgmAllocProc: (0x205e390) allocated [ CSSD]2013-05-23 16:10:07.998 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:10:07.998 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e390 - 1,2,3 [ CSSD]2013-05-23 16:10:07.998 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dfee0) proc(0x205e390) pid(2306) version 11:1:1:4 [ CSSD]2013-05-23 16:10:08.017 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:08.017 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:08.557 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e390 [ CSSD]2013-05-23 16:10:08.557 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e390) con(0x22dfee0) skgpid ospid 2306 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:09.219 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:10:09.219 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1211 (0x7f431bfe4d90) [ CSSD]2013-05-23 16:10:09.220 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4d90 (RPC#1211) tag(4bb002c) sent to node 1 [ CSSD]2013-05-23 16:10:09.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:09.220 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1211 (0x7f431bfe4d90) [ CSSD]2013-05-23 16:10:09.220 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4d90 (RPC#1211) state 6, flags 0x202 [ CSSD]2013-05-23 16:10:09.220 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4d90, ret 0, client (nil) [ CSSD]2013-05-23 16:10:09.220 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1211 [ CSSD]2013-05-23 16:10:09.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:09.406 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:09.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:09.489 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014390, gin 1, member 0 [ CSSD]2013-05-23 16:10:09.489 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:10:09.489 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:10:09.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:09.496 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:10:09.496 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:10:09.561 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:11.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:11.412 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:11.568 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:12.020 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:12.020 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:10:12.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:12.670 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014391, gin 1, member 0 [ CSSD]2013-05-23 16:10:12.670 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:10:12.670 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:10:12.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:12.676 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:10:12.676 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:10:13.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:13.418 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:13.619 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:14.158 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:10:14.159 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:10:14.160 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:10:14.160 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c147a00) proc(0x7f431c147f40) pid(2450) version 11:1:1:4 [ CSSD]2013-05-23 16:10:14.166 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:10:14.166 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:10:14.166 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:10:14.166 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:14.166 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1212 (0x7f431bfe4e10) [ CSSD]2013-05-23 16:10:14.166 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4e10 (RPC#1212) tag(4bc002c) sent to node 1 [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04bc002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014392, gin 1, member 0 [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4e10 (RPC#1212) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4e10, ret 0, client 0x7f431c121510 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:10:14.167 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1212 [ CSSD]2013-05-23 16:10:14.174 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:10:14.174 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:14.174 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:14.174 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1213 (0x7f431bfe4e90) [ CSSD]2013-05-23 16:10:14.175 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4e90 (RPC#1213) tag(4bd002c) sent to node 1 [ CSSD]2013-05-23 16:10:14.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04bd002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:14.175 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:14.175 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4e90 (RPC#1213) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:14.175 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4e90, ret 0, client 0x7f431c121510 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:10:14.175 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1213 [ CSSD]2013-05-23 16:10:14.175 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:10:14.377 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:10:14.378 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c147a00) skgpid ospid 2450 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:15.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:15.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:15.626 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:16.482 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:10:16.483 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:10:16.483 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:10:16.483 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c147f40) pid(2504) version 11:1:1:4 [ CSSD]2013-05-23 16:10:16.489 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c147f40), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:10:16.489 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:10:16.489 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:10:16.489 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:16.489 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1214 (0x7f431bfe4f10) [ CSSD]2013-05-23 16:10:16.489 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4f10 (RPC#1214) tag(4be002c) sent to node 1 [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04be002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014393, gin 1, member 0 [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4f10 (RPC#1214) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4f10, ret 0, client 0x7f431c0e5910 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:10:16.490 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1214 [ CSSD]2013-05-23 16:10:16.496 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:10:16.496 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:16.496 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:16.496 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1215 (0x7f431bfe4f90) [ CSSD]2013-05-23 16:10:16.497 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4f90 (RPC#1215) tag(4bf002c) sent to node 1 [ CSSD]2013-05-23 16:10:16.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04bf002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:16.497 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:16.497 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4f90 (RPC#1215) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:16.497 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4f90, ret 0, client 0x7f431c0e5910 member 0x7f431c13eea0 [ CSSD]2013-05-23 16:10:16.497 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1215 [ CSSD]2013-05-23 16:10:16.497 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:10:16.571 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:10:16.572 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:10:16.572 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:10:16.572 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1348f0) pid(1362/1362) version 11:1:1:4 [ CSSD]2013-05-23 16:10:16.578 [1191196992] >TRACE: clssgmRegisterClient: proc(65/0x7f431c1348f0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:10:16.578 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:10:16.578 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:10:16.578 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 36 [ CSSD]2013-05-23 16:10:16.700 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:10:16.700 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c0e6720) skgpid ospid 2504 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:16.795 [1191196992] >TRACE: clssgmRegisterClient: proc(65/0x7f431c1348f0), client(2/0x7f431c0e5910) [ CSSD]2013-05-23 16:10:16.795 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0e5910) [ CSSD]2013-05-23 16:10:16.795 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:10:16.795 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 28 [ CSSD]2013-05-23 16:10:17.024 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:17.024 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:17.476 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:17.476 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:17.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:18.986 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c091450) [ CSSD]2013-05-23 16:10:18.986 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c091450), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:10:18.986 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c091450), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:10:18.986 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 35 [ CSSD]2013-05-23 16:10:18.986 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090ee0 [ CSSD]2013-05-23 16:10:18.986 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090ee0) con(0x7f431c108e60) skgpid 29669 ospid 29669 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:18.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c13b7e0) [ CSSD]2013-05-23 16:10:18.989 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c13b7e0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:10:18.989 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c13b7e0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:10:18.989 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 27 [ CSSD]2013-05-23 16:10:18.995 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c13b2f0) [ CSSD]2013-05-23 16:10:18.995 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c13b2f0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:10:18.995 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c13b2f0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:10:18.995 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 34 [ CSSD]2013-05-23 16:10:18.995 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ad10 [ CSSD]2013-05-23 16:10:18.995 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ad10) con(0x7f431c0915c0) skgpid 29667 ospid 29667 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:19.243 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:10:19.244 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1216 (0x7f431bfe5010) [ CSSD]2013-05-23 16:10:19.244 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5010 (RPC#1216) tag(4c0002c) sent to node 1 [ CSSD]2013-05-23 16:10:19.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:19.244 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1216 (0x7f431bfe5010) [ CSSD]2013-05-23 16:10:19.244 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5010 (RPC#1216) state 6, flags 0x202 [ CSSD]2013-05-23 16:10:19.244 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5010, ret 0, client (nil) [ CSSD]2013-05-23 16:10:19.245 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1216 [ CSSD]2013-05-23 16:10:19.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:19.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:19.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:21.493 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:21.493 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:21.650 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:22.027 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:22.027 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:23.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:23.499 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:23.658 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:24.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:24.738 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014394, gin 1, member 0 [ CSSD]2013-05-23 16:10:24.739 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:10:24.739 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:10:24.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:24.745 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:10:24.745 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:10:25.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:25.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:25.665 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:27.030 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:27.031 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:27.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:27.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:27.717 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:29.286 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:10:29.287 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1217 (0x7f431bfe5090) [ CSSD]2013-05-23 16:10:29.287 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5090 (RPC#1217) tag(4c1002c) sent to node 1 [ CSSD]2013-05-23 16:10:29.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:29.287 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1217 (0x7f431bfe5090) [ CSSD]2013-05-23 16:10:29.287 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5090 (RPC#1217) state 6, flags 0x202 [ CSSD]2013-05-23 16:10:29.287 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5090, ret 0, client (nil) [ CSSD]2013-05-23 16:10:29.287 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1217 [ CSSD]2013-05-23 16:10:29.413 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c081e40) allocated [ CSSD]2013-05-23 16:10:29.413 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:10:29.414 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c081e40 - 1,2,3 [ CSSD]2013-05-23 16:10:29.414 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c081e40) pid(2610) version 11:1:1:4 [ CSSD]2013-05-23 16:10:29.420 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c081e40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:10:29.420 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:10:29.420 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6b70, requested num -1 [ CSSD]2013-05-23 16:10:29.420 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:29.420 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1218 (0x7f431bfe5110) [ CSSD]2013-05-23 16:10:29.420 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5110 (RPC#1218) tag(4c2002c) sent to node 1 [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04c2002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014395, gin 1, member 0 [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090cf0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5110 (RPC#1218) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5110, ret 0, client 0x7f431c121510 member 0x7f431c091450 [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:10:29.421 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1218 [ CSSD]2013-05-23 16:10:29.427 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:10:29.427 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:29.427 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c091450) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:29.427 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1219 (0x7f431bfe5190) [ CSSD]2013-05-23 16:10:29.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5190 (RPC#1219) tag(4c3002c) sent to node 1 [ CSSD]2013-05-23 16:10:29.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04c3002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:29.428 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:29.428 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5190 (RPC#1219) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:29.428 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5190, ret 0, client 0x7f431c121510 member 0x7f431c091450 [ CSSD]2013-05-23 16:10:29.428 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1219 [ CSSD]2013-05-23 16:10:29.428 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c091450) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:10:29.631 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c081e40 [ CSSD]2013-05-23 16:10:29.631 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c081e40) con(0x7f431c11ab50) skgpid ospid 2610 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:29.638 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147f40) allocated [ CSSD]2013-05-23 16:10:29.639 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:10:29.639 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147f40 - 1,2,3 [ CSSD]2013-05-23 16:10:29.639 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c147f40) pid(2545/2545) version 11:1:1:4 [ CSSD]2013-05-23 16:10:29.655 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c147f40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:10:29.656 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:10:29.656 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:10:29.656 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 35 [ CSSD]2013-05-23 16:10:29.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:29.668 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:29.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:31.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:31.674 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:31.748 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:32.034 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:32.034 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:33.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:33.680 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:33.760 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:35.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:35.691 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:35.767 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:37.038 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:37.038 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:37.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:37.696 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:37.817 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:39.311 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:10:39.311 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1220 (0x7f431bfe5210) [ CSSD]2013-05-23 16:10:39.311 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5210 (RPC#1220) tag(4c4002c) sent to node 1 [ CSSD]2013-05-23 16:10:39.312 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:39.312 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1220 (0x7f431bfe5210) [ CSSD]2013-05-23 16:10:39.312 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5210 (RPC#1220) state 6, flags 0x202 [ CSSD]2013-05-23 16:10:39.312 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5210, ret 0, client (nil) [ CSSD]2013-05-23 16:10:39.312 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1220 [ CSSD]2013-05-23 16:10:39.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:39.768 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:39.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:39.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:39.999 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014396, gin 1, member 0 [ CSSD]2013-05-23 16:10:40.000 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:10:40.000 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:10:40.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:40.006 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:10:40.006 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:10:41.831 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:41.833 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:41.833 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:42.042 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:42.042 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:43.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:43.867 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:43.920 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:44.667 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 16:10:44.668 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:10:44.669 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 16:10:44.669 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13b8a0) proc(0x7f431c132e60) pid(2755) version 11:1:1:4 [ CSSD]2013-05-23 16:10:44.675 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c132e60), client(1/0x7f431c0915d0) [ CSSD]2013-05-23 16:10:44.675 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:10:44.675 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0915d0 with con 0x7f431c1216a0, requested num -1 [ CSSD]2013-05-23 16:10:44.675 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:44.675 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1221 (0x7f431bfe5290) [ CSSD]2013-05-23 16:10:44.676 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5290 (RPC#1221) tag(4c5002c) sent to node 1 [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04c5002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014397, gin 1, member 0 [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5290 (RPC#1221) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5290, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915d0 [ CSSD]2013-05-23 16:10:44.676 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1221 [ CSSD]2013-05-23 16:10:44.682 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:10:44.682 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:44.682 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:44.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1222 (0x7f431bfe5310) [ CSSD]2013-05-23 16:10:44.682 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5310 (RPC#1222) tag(4c6002c) sent to node 1 [ CSSD]2013-05-23 16:10:44.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04c6002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:44.683 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:44.683 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5310 (RPC#1222) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:44.683 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5310, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:10:44.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1222 [ CSSD]2013-05-23 16:10:44.683 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:10:44.885 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 16:10:44.885 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c13b8a0) skgpid ospid 2755 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:45.872 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:45.872 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:45.927 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:46.044 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:46.045 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:10:47.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:47.897 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:47.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:49.335 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:10:49.336 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1223 (0x7f431bfe5390) [ CSSD]2013-05-23 16:10:49.336 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5390 (RPC#1223) tag(4c7002c) sent to node 1 [ CSSD]2013-05-23 16:10:49.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:49.336 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1223 (0x7f431bfe5390) [ CSSD]2013-05-23 16:10:49.336 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5390 (RPC#1223) state 6, flags 0x202 [ CSSD]2013-05-23 16:10:49.336 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5390, ret 0, client (nil) [ CSSD]2013-05-23 16:10:49.336 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1223 [ CSSD]2013-05-23 16:10:49.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:49.908 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:49.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:51.048 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:51.049 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:51.914 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:51.914 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:51.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:53.967 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:53.967 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:54.040 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:55.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:55.249 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014398, gin 1, member 0 [ CSSD]2013-05-23 16:10:55.249 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:10:55.249 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:10:55.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:55.255 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:10:55.255 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:10:55.973 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:55.973 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:56.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:56.052 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:10:56.052 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:10:56.287 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 16:10:56.289 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:10:56.289 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 16:10:56.289 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6b70) proc(0x7f431c132e60) pid(2883) version 11:1:1:4 [ CSSD]2013-05-23 16:10:56.295 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c132e60), client(1/0x7f431c0915d0) [ CSSD]2013-05-23 16:10:56.295 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:10:56.295 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0915d0 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:10:56.295 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:56.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1224 (0x7f431bfe5410) [ CSSD]2013-05-23 16:10:56.295 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5410 (RPC#1224) tag(4c8002c) sent to node 1 [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04c8002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014399, gin 1, member 0 [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5410 (RPC#1224) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5410, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915d0 [ CSSD]2013-05-23 16:10:56.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1224 [ CSSD]2013-05-23 16:10:56.303 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:10:56.303 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:56.303 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:56.303 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1225 (0x7f431bfe5490) [ CSSD]2013-05-23 16:10:56.303 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5490 (RPC#1225) tag(4c9002c) sent to node 1 [ CSSD]2013-05-23 16:10:56.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04c9002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:56.303 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:56.303 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5490 (RPC#1225) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:56.303 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5490, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:10:56.303 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1225 [ CSSD]2013-05-23 16:10:56.303 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:10:56.506 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 16:10:56.506 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c0e6b70) skgpid ospid 2883 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:10:57.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:57.978 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:10:58.058 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:10:59.360 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:10:59.361 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1226 (0x7f431bfe5510) [ CSSD]2013-05-23 16:10:59.361 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5510 (RPC#1226) tag(4ca002c) sent to node 1 [ CSSD]2013-05-23 16:10:59.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:59.361 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1226 (0x7f431bfe5510) [ CSSD]2013-05-23 16:10:59.361 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5510 (RPC#1226) state 6, flags 0x202 [ CSSD]2013-05-23 16:10:59.361 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5510, ret 0, client (nil) [ CSSD]2013-05-23 16:10:59.361 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1226 [ CSSD]2013-05-23 16:10:59.923 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 16:10:59.924 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:10:59.925 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 16:10:59.925 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13b8a0) proc(0x7f431c132e60) pid(2913) version 11:1:1:4 [ CSSD]2013-05-23 16:10:59.931 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c132e60), client(1/0x7f431c0915d0) [ CSSD]2013-05-23 16:10:59.931 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:10:59.931 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0915d0 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:10:59.931 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:59.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1227 (0x7f431bfe5590) [ CSSD]2013-05-23 16:10:59.931 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5590 (RPC#1227) tag(4cb002c) sent to node 1 [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04cb002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014400, gin 1, member 0 [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133560/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5590 (RPC#1227) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5590, ret 0, client 0x7f431c0915d0 member 0x7f431c133860 [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915d0 [ CSSD]2013-05-23 16:10:59.932 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1227 [ CSSD]2013-05-23 16:10:59.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:10:59.939 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:59.939 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133860) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:10:59.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1228 (0x7f431bfe5610) [ CSSD]2013-05-23 16:10:59.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5610 (RPC#1228) tag(4cc002c) sent to node 1 [ CSSD]2013-05-23 16:10:59.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04cc002c) incarnation(174631051) [ CSSD]2013-05-23 16:10:59.939 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:10:59.939 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5610 (RPC#1228) state 6, flags 0x100 [ CSSD]2013-05-23 16:10:59.939 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5610, ret 0, client 0x7f431c0915d0 member 0x7f431c133860 [ CSSD]2013-05-23 16:10:59.939 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1228 [ CSSD]2013-05-23 16:10:59.939 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133860) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:10:59.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:10:59.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:00.064 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:00.142 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 16:11:00.142 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c13b8a0) skgpid ospid 2913 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:01.055 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:01.055 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:02.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:02.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:02.071 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:04.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:04.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:04.091 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:05.058 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:05.058 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:11:06.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:06.023 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:06.097 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:08.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:08.029 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:08.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:09.384 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:11:09.385 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1229 (0x7f431bfe5690) [ CSSD]2013-05-23 16:11:09.385 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5690 (RPC#1229) tag(4cd002c) sent to node 1 [ CSSD]2013-05-23 16:11:09.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:09.385 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1229 (0x7f431bfe5690) [ CSSD]2013-05-23 16:11:09.385 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5690 (RPC#1229) state 6, flags 0x202 [ CSSD]2013-05-23 16:11:09.385 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5690, ret 0, client (nil) [ CSSD]2013-05-23 16:11:09.385 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1229 [ CSSD]2013-05-23 16:11:10.062 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:10.062 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:10.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:10.080 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:10.155 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:10.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:10.497 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014401, gin 1, member 0 [ CSSD]2013-05-23 16:11:10.497 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:11:10.497 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:11:10.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:10.504 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:11:10.504 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:11:12.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:12.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:12.162 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:12.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:12.928 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014402, gin 1, member 0 [ CSSD]2013-05-23 16:11:12.928 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:11:12.928 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:11:12.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:12.934 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:11:12.934 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:11:14.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:14.117 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:14.173 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:15.066 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:15.066 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:15.178 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 16:11:15.179 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:11:15.180 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 16:11:15.180 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c132e60) pid(3087) version 11:1:1:4 [ CSSD]2013-05-23 16:11:15.186 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c132e60), client(1/0x7f431c0915d0) [ CSSD]2013-05-23 16:11:15.186 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:11:15.186 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0915d0 with con 0x7f431c13b8a0, requested num -1 [ CSSD]2013-05-23 16:11:15.186 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:15.186 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1230 (0x7f431bfe5710) [ CSSD]2013-05-23 16:11:15.186 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5710 (RPC#1230) tag(4ce002c) sent to node 1 [ CSSD]2013-05-23 16:11:15.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ce002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:15.186 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014403, gin 1, member 0 [ CSSD]2013-05-23 16:11:15.186 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:11:15.187 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:11:15.187 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:11:15.187 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5710 (RPC#1230) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:15.187 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5710, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:11:15.187 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915d0 [ CSSD]2013-05-23 16:11:15.187 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1230 [ CSSD]2013-05-23 16:11:15.194 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:11:15.194 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:15.194 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:15.194 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1231 (0x7f431bfe5790) [ CSSD]2013-05-23 16:11:15.194 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5790 (RPC#1231) tag(4cf002c) sent to node 1 [ CSSD]2013-05-23 16:11:15.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04cf002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:15.194 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:15.194 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5790 (RPC#1231) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:15.194 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5790, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:11:15.195 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1231 [ CSSD]2013-05-23 16:11:15.195 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:11:15.397 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 16:11:15.397 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c0e6e20) skgpid ospid 3087 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:16.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:16.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:16.180 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:16.693 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 16:11:16.694 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:11:16.695 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 16:11:16.695 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6b70) proc(0x7f431c132e60) pid(3146) version 11:1:1:4 [ CSSD]2013-05-23 16:11:16.702 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c132e60), client(1/0x7f431c0915d0) [ CSSD]2013-05-23 16:11:16.702 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:11:16.702 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0915d0 with con 0x7f431c1216a0, requested num -1 [ CSSD]2013-05-23 16:11:16.702 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:11:16.702 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1232 (0x7f431bfe5810) [ CSSD]2013-05-23 16:11:16.702 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5810 (RPC#1232) tag(4d0002c) sent to node 1 [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d0002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014404, gin 1, member 0 [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133540/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5810 (RPC#1232) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5810, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915d0 [ CSSD]2013-05-23 16:11:16.703 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1232 [ CSSD]2013-05-23 16:11:16.709 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:11:16.709 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915d0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:11:16.709 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133840) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:11:16.709 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1233 (0x7f431bfe5890) [ CSSD]2013-05-23 16:11:16.709 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5890 (RPC#1233) tag(4d1002c) sent to node 1 [ CSSD]2013-05-23 16:11:16.709 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04d1002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:16.709 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:11:16.709 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5890 (RPC#1233) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:16.709 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5890, ret 0, client 0x7f431c0915d0 member 0x7f431c133840 [ CSSD]2013-05-23 16:11:16.710 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1233 [ CSSD]2013-05-23 16:11:16.710 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c133840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:11:16.742 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0837d0) allocated [ CSSD]2013-05-23 16:11:16.743 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:11:16.743 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0837d0 - 1,2,3 [ CSSD]2013-05-23 16:11:16.743 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13b8a0) proc(0x7f431c0837d0) pid(3152) version 11:1:1:4 [ CSSD]2013-05-23 16:11:16.750 [1191196992] >TRACE: clssgmRegisterClient: proc(65/0x7f431c0837d0), client(1/0x7f431c0915d0) [ CSSD]2013-05-23 16:11:16.750 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:11:16.750 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0915d0 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:11:16.750 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:16.750 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1234 (0x7f431bfe5910) [ CSSD]2013-05-23 16:11:16.750 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5910 (RPC#1234) tag(4d2002c) sent to node 1 [ CSSD]2013-05-23 16:11:16.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d2002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:16.750 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014405, gin 1, member 0 [ CSSD]2013-05-23 16:11:16.750 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134510/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:11:16.751 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:11:16.751 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:11:16.751 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5910 (RPC#1234) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:16.751 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5910, ret 0, client 0x7f431c0915d0 member 0x7f431c083690 [ CSSD]2013-05-23 16:11:16.751 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915d0 [ CSSD]2013-05-23 16:11:16.751 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1234 [ CSSD]2013-05-23 16:11:16.757 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915d0) [ CSSD]2013-05-23 16:11:16.758 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:16.758 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083690) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:16.758 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1235 (0x7f431bfe5990) [ CSSD]2013-05-23 16:11:16.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5990 (RPC#1235) tag(4d3002c) sent to node 1 [ CSSD]2013-05-23 16:11:16.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04d3002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:16.758 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:16.758 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5990 (RPC#1235) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:16.758 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5990, ret 0, client 0x7f431c0915d0 member 0x7f431c083690 [ CSSD]2013-05-23 16:11:16.758 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1235 [ CSSD]2013-05-23 16:11:16.758 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083690) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:11:16.823 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c14a110) allocated [ CSSD]2013-05-23 16:11:16.823 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:11:16.824 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c14a110 - 1,2,3 [ CSSD]2013-05-23 16:11:16.824 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c14a110) pid(3156) version 11:1:1:4 [ CSSD]2013-05-23 16:11:16.838 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:11:16.839 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:11:16.841 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c14a110 [ CSSD]2013-05-23 16:11:16.841 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c14a110) con(0x7f431c11ab50) skgpid ospid 3156 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:16.912 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 16:11:16.912 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c0e6b70) skgpid ospid 3146 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:16.961 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0837d0 [ CSSD]2013-05-23 16:11:16.961 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0837d0) con(0x7f431c13b8a0) skgpid ospid 3152 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:18.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:18.126 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:18.199 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:19.069 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:19.069 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:11:19.410 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:11:19.411 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1236 (0x7f431bfe5a10) [ CSSD]2013-05-23 16:11:19.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5a10 (RPC#1236) tag(4d4002c) sent to node 1 [ CSSD]2013-05-23 16:11:19.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:19.411 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1236 (0x7f431bfe5a10) [ CSSD]2013-05-23 16:11:19.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5a10 (RPC#1236) state 6, flags 0x202 [ CSSD]2013-05-23 16:11:19.411 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5a10, ret 0, client (nil) [ CSSD]2013-05-23 16:11:19.412 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1236 [ CSSD]2013-05-23 16:11:20.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:20.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:20.206 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:22.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:22.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:22.212 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:23.281 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c081f70) [ CSSD]2013-05-23 16:11:23.281 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c081f70), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:11:23.281 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c081f70), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:11:23.281 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c137c20) [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c137c20), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c137c20), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 34 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092880 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092880) con(0x7f431c091c00) skgpid 30361 ospid 30361 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c144a40) [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c144a40), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c144a40), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:11:23.288 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:11:23.293 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c1509e0) [ CSSD]2013-05-23 16:11:23.293 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c1509e0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:11:23.293 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c1509e0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:11:23.293 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 16:11:23.293 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1504a0 [ CSSD]2013-05-23 16:11:23.293 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1504a0) con(0x7f431c13bee0) skgpid 30359 ospid 30359 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:24.073 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:24.073 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:24.189 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:24.189 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:24.243 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:25.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:25.748 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014406, gin 1, member 0 [ CSSD]2013-05-23 16:11:25.748 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:11:25.748 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:11:25.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:25.755 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:11:25.755 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:11:26.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:26.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:26.250 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:28.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:28.220 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:28.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:29.077 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:29.077 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:29.437 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:11:29.438 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1237 (0x7f431bfe5a90) [ CSSD]2013-05-23 16:11:29.438 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5a90 (RPC#1237) tag(4d5002c) sent to node 1 [ CSSD]2013-05-23 16:11:29.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:29.438 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1237 (0x7f431bfe5a90) [ CSSD]2013-05-23 16:11:29.438 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5a90 (RPC#1237) state 6, flags 0x202 [ CSSD]2013-05-23 16:11:29.438 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5a90, ret 0, client (nil) [ CSSD]2013-05-23 16:11:29.439 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1237 [ CSSD]2013-05-23 16:11:30.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:30.230 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:30.267 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:30.436 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:11:30.438 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:11:30.438 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:11:30.438 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c137b90) pid(3309) version 11:1:1:4 [ CSSD]2013-05-23 16:11:30.444 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:11:30.444 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:11:30.444 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:11:30.444 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:30.444 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1238 (0x7f431bfe5b10) [ CSSD]2013-05-23 16:11:30.444 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5b10 (RPC#1238) tag(4d6002c) sent to node 1 [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d6002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014407, gin 1, member 0 [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5b10 (RPC#1238) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5b10, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:11:30.445 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1238 [ CSSD]2013-05-23 16:11:30.452 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:11:30.452 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:30.452 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:30.452 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1239 (0x7f431bfe5b90) [ CSSD]2013-05-23 16:11:30.452 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5b90 (RPC#1239) tag(4d7002c) sent to node 1 [ CSSD]2013-05-23 16:11:30.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04d7002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:30.452 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:30.452 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5b90 (RPC#1239) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:30.452 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5b90, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:11:30.453 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1239 [ CSSD]2013-05-23 16:11:30.453 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:11:30.655 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:11:30.655 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c0e6e20) skgpid ospid 3309 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:32.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:32.236 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:32.273 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:34.080 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:34.081 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:34.289 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:34.289 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:34.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:36.294 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:36.294 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:36.310 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:38.084 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:38.084 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:11:38.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:38.299 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:38.341 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:39.464 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:11:39.465 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1240 (0x7f431bfe5c10) [ CSSD]2013-05-23 16:11:39.465 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5c10 (RPC#1240) tag(4d8002c) sent to node 1 [ CSSD]2013-05-23 16:11:39.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:39.465 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1240 (0x7f431bfe5c10) [ CSSD]2013-05-23 16:11:39.465 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5c10 (RPC#1240) state 6, flags 0x202 [ CSSD]2013-05-23 16:11:39.465 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5c10, ret 0, client (nil) [ CSSD]2013-05-23 16:11:39.465 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1240 [ CSSD]2013-05-23 16:11:40.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:40.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:40.348 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:41.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:41.008 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014408, gin 1, member 0 [ CSSD]2013-05-23 16:11:41.008 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:11:41.008 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:11:41.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:41.048 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:11:41.048 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:11:42.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:42.336 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:42.358 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:43.087 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:43.088 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:44.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:44.389 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:44.405 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:45.688 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:11:45.689 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:11:45.689 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:11:45.689 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137b90) pid(3467) version 11:1:1:4 [ CSSD]2013-05-23 16:11:45.695 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:11:45.695 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:11:45.695 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:11:45.695 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:45.695 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1241 (0x7f431bfe5c90) [ CSSD]2013-05-23 16:11:45.695 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5c90 (RPC#1241) tag(4d9002c) sent to node 1 [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d9002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014409, gin 1, member 0 [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5c90 (RPC#1241) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5c90, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:11:45.696 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1241 [ CSSD]2013-05-23 16:11:45.702 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:11:45.702 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:45.702 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:11:45.702 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1242 (0x7f431bfe5d10) [ CSSD]2013-05-23 16:11:45.702 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5d10 (RPC#1242) tag(4da002c) sent to node 1 [ CSSD]2013-05-23 16:11:45.702 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04da002c) incarnation(174631051) [ CSSD]2013-05-23 16:11:45.702 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:11:45.702 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5d10 (RPC#1242) state 6, flags 0x100 [ CSSD]2013-05-23 16:11:45.702 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5d10, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:11:45.702 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1242 [ CSSD]2013-05-23 16:11:45.703 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:11:45.905 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:11:45.905 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c11ab50) skgpid ospid 3467 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:11:46.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:46.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:46.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:48.091 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:48.092 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:48.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:48.400 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:48.441 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:49.488 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:11:49.488 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1243 (0x7f431bfe5d90) [ CSSD]2013-05-23 16:11:49.488 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5d90 (RPC#1243) tag(4db002c) sent to node 1 [ CSSD]2013-05-23 16:11:49.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:49.489 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1243 (0x7f431bfe5d90) [ CSSD]2013-05-23 16:11:49.489 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5d90 (RPC#1243) state 6, flags 0x202 [ CSSD]2013-05-23 16:11:49.489 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5d90, ret 0, client (nil) [ CSSD]2013-05-23 16:11:49.489 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1243 [ CSSD]2013-05-23 16:11:50.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:50.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:50.448 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:52.094 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:52.095 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:11:52.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:52.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:52.458 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:54.463 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:54.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:54.490 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:56.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:56.259 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014410, gin 1, member 0 [ CSSD]2013-05-23 16:11:56.259 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:11:56.259 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:11:56.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:56.266 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:11:56.266 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:11:56.470 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:56.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:56.495 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:57.098 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:11:57.098 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:11:58.482 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:11:58.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:58.520 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:11:59.513 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:11:59.513 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1244 (0x7f431bfe5e10) [ CSSD]2013-05-23 16:11:59.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5e10 (RPC#1244) tag(4dc002c) sent to node 1 [ CSSD]2013-05-23 16:11:59.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:11:59.514 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1244 (0x7f431bfe5e10) [ CSSD]2013-05-23 16:11:59.514 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5e10 (RPC#1244) state 6, flags 0x202 [ CSSD]2013-05-23 16:11:59.514 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5e10, ret 0, client (nil) [ CSSD]2013-05-23 16:11:59.514 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1244 [ CSSD]2013-05-23 16:12:00.489 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:00.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:00.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:00.942 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:00.943 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:12:00.943 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:00.943 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137b90) pid(3622) version 11:1:1:4 [ CSSD]2013-05-23 16:12:00.950 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:00.950 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:00.950 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6b70, requested num -1 [ CSSD]2013-05-23 16:12:00.950 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:00.950 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1245 (0x7f431bfe5e90) [ CSSD]2013-05-23 16:12:00.950 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5e90 (RPC#1245) tag(4dd002c) sent to node 1 [ CSSD]2013-05-23 16:12:00.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04dd002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:00.950 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014411, gin 1, member 0 [ CSSD]2013-05-23 16:12:00.950 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:12:00.951 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:00.951 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:12:00.951 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5e90 (RPC#1245) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:00.951 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5e90, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:00.951 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:00.951 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1245 [ CSSD]2013-05-23 16:12:00.957 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:00.957 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:00.957 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:00.957 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1246 (0x7f431bfe5f10) [ CSSD]2013-05-23 16:12:00.957 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5f10 (RPC#1246) tag(4de002c) sent to node 1 [ CSSD]2013-05-23 16:12:00.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04de002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:00.958 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:00.958 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5f10 (RPC#1246) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:00.958 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5f10, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:00.958 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1246 [ CSSD]2013-05-23 16:12:00.958 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:01.160 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:01.160 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c11ab50) skgpid ospid 3622 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:02.101 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:02.101 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:02.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:02.535 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:02.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:04.586 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:04.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:04.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:06.338 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:06.340 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:12:06.340 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:06.340 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137b90) pid(3702) version 11:1:1:4 [ CSSD]2013-05-23 16:12:06.347 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:06.347 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:06.347 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:12:06.347 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:06.347 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1247 (0x7f431bfe5f90) [ CSSD]2013-05-23 16:12:06.347 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5f90 (RPC#1247) tag(4df002c) sent to node 1 [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04df002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014412, gin 1, member 0 [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5f90 (RPC#1247) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5f90, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:06.347 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:06.348 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1247 [ CSSD]2013-05-23 16:12:06.354 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:06.354 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:06.354 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:06.354 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1248 (0x7f431bfe6010) [ CSSD]2013-05-23 16:12:06.354 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6010 (RPC#1248) tag(4e0002c) sent to node 1 [ CSSD]2013-05-23 16:12:06.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04e0002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:06.354 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:06.354 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6010 (RPC#1248) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:06.354 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6010, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:06.355 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1248 [ CSSD]2013-05-23 16:12:06.355 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:06.557 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:06.557 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c11ab50) skgpid ospid 3702 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:06.591 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:06.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:06.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:07.105 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:07.105 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:08.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:08.623 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:08.642 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:09.539 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:12:09.540 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1249 (0x7f431bfe6090) [ CSSD]2013-05-23 16:12:09.540 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6090 (RPC#1249) tag(4e1002c) sent to node 1 [ CSSD]2013-05-23 16:12:09.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:09.540 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1249 (0x7f431bfe6090) [ CSSD]2013-05-23 16:12:09.540 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6090 (RPC#1249) state 6, flags 0x202 [ CSSD]2013-05-23 16:12:09.540 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6090, ret 0, client (nil) [ CSSD]2013-05-23 16:12:09.541 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1249 [ CSSD]2013-05-23 16:12:10.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:10.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:10.653 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:11.108 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:11.108 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:12:11.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:11.508 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014413, gin 1, member 0 [ CSSD]2013-05-23 16:12:11.508 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:12:11.508 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:12:11.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:11.513 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:12:11.513 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:12:12.658 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:12.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:12.659 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:13.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:13.197 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014414, gin 1, member 0 [ CSSD]2013-05-23 16:12:13.197 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:12:13.197 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:12:13.203 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:13.203 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:12:13.203 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:12:14.664 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:14.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:14.691 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:15.130 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:15.131 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:12:15.132 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:15.132 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137b90) pid(3998) version 11:1:1:4 [ CSSD]2013-05-23 16:12:15.256 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13a9e0) allocated [ CSSD]2013-05-23 16:12:15.258 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:12:15.258 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13a9e0 - 1,2,3 [ CSSD]2013-05-23 16:12:15.258 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6b70) proc(0x7f431c13a9e0) pid(4008) version 11:1:1:4 [ CSSD]2013-05-23 16:12:15.275 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:12:15.277 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:12:15.279 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13a9e0 [ CSSD]2013-05-23 16:12:15.279 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13a9e0) con(0x7f431c0e6b70) skgpid ospid 4008 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:15.356 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(65/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:15.356 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 65 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:15.356 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:12:15.356 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:12:15.356 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1250 (0x7f431bfe6110) [ CSSD]2013-05-23 16:12:15.356 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6110 (RPC#1250) tag(4e2002c) sent to node 1 [ CSSD]2013-05-23 16:12:15.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04e2002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:15.356 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014415, gin 1, member 0 [ CSSD]2013-05-23 16:12:15.356 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c13bab0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:12:15.357 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:15.357 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:12:15.357 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6110 (RPC#1250) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:15.357 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6110, ret 0, client 0x7f431c12ba00 member 0x7f431c13bdb0 [ CSSD]2013-05-23 16:12:15.357 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:15.357 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1250 [ CSSD]2013-05-23 16:12:15.363 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 65 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:15.363 [1191196992] >TRACE: clssgmExitGrock: client 65 (0x7f431c12ba00), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:12:15.363 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13bdb0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:12:15.363 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1251 (0x7f431bfe6190) [ CSSD]2013-05-23 16:12:15.363 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6190 (RPC#1251) tag(4e3002c) sent to node 1 [ CSSD]2013-05-23 16:12:15.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04e3002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:15.364 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:12:15.364 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6190 (RPC#1251) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:15.364 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6190, ret 0, client 0x7f431c12ba00 member 0x7f431c13bdb0 [ CSSD]2013-05-23 16:12:15.364 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1251 [ CSSD]2013-05-23 16:12:15.364 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c13bdb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:15.365 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:15.365 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c11ab50) skgpid ospid 3998 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:16.112 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:16.112 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:16.187 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:16.189 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:12:16.189 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:16.189 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6b70) proc(0x7f431c137b90) pid(4024) version 11:1:1:4 [ CSSD]2013-05-23 16:12:16.196 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:16.196 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:16.196 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:12:16.196 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:16.196 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1252 (0x7f431bfe6210) [ CSSD]2013-05-23 16:12:16.196 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6210 (RPC#1252) tag(4e4002c) sent to node 1 [ CSSD]2013-05-23 16:12:16.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04e4002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:16.196 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014416, gin 1, member 0 [ CSSD]2013-05-23 16:12:16.196 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:12:16.197 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:16.197 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:12:16.197 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6210 (RPC#1252) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:16.197 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6210, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:16.197 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:16.197 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1252 [ CSSD]2013-05-23 16:12:16.203 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:16.203 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:16.203 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:16.203 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1253 (0x7f431bfe6290) [ CSSD]2013-05-23 16:12:16.203 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6290 (RPC#1253) tag(4e5002c) sent to node 1 [ CSSD]2013-05-23 16:12:16.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04e5002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:16.204 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:16.204 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6290 (RPC#1253) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:16.204 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6290, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:16.204 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1253 [ CSSD]2013-05-23 16:12:16.204 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:16.406 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:16.406 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c0e6b70) skgpid ospid 4024 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:16.671 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:16.716 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:16.716 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:17.001 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:17.002 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:12:17.002 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:17.002 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c137b90) pid(4071) version 11:1:1:4 [ CSSD]2013-05-23 16:12:17.008 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:17.008 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:17.008 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6b70, requested num -1 [ CSSD]2013-05-23 16:12:17.008 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:17.008 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1254 (0x7f431bfe6310) [ CSSD]2013-05-23 16:12:17.009 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6310 (RPC#1254) tag(4e6002c) sent to node 1 [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04e6002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014417, gin 1, member 0 [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6310 (RPC#1254) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6310, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:17.009 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1254 [ CSSD]2013-05-23 16:12:17.016 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:17.016 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:17.016 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:17.016 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1255 (0x7f431bfe6390) [ CSSD]2013-05-23 16:12:17.017 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6390 (RPC#1255) tag(4e7002c) sent to node 1 [ CSSD]2013-05-23 16:12:17.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04e7002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:17.017 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:17.017 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6390 (RPC#1255) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:17.017 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6390, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:17.017 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1255 [ CSSD]2013-05-23 16:12:17.017 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:17.220 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:17.220 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c0e6e20) skgpid ospid 4071 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:18.683 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:18.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:18.722 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:19.563 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:12:19.563 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1256 (0x7f431bfe6410) [ CSSD]2013-05-23 16:12:19.563 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6410 (RPC#1256) tag(4e8002c) sent to node 1 [ CSSD]2013-05-23 16:12:19.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:19.564 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1256 (0x7f431bfe6410) [ CSSD]2013-05-23 16:12:19.564 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6410 (RPC#1256) state 6, flags 0x202 [ CSSD]2013-05-23 16:12:19.564 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6410, ret 0, client (nil) [ CSSD]2013-05-23 16:12:19.564 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1256 [ CSSD]2013-05-23 16:12:20.689 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:20.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:20.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:21.115 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:21.116 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:22.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:22.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:22.740 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:24.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:24.762 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:24.766 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:26.119 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:26.119 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:26.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:26.756 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014418, gin 1, member 0 [ CSSD]2013-05-23 16:12:26.756 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:12:26.756 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:12:26.772 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:26.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:26.772 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:12:26.772 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:12:26.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:26.772 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:28.784 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:28.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:28.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:29.590 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:12:29.590 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1257 (0x7f431bfe6490) [ CSSD]2013-05-23 16:12:29.590 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6490 (RPC#1257) tag(4e9002c) sent to node 1 [ CSSD]2013-05-23 16:12:29.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:29.591 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1257 (0x7f431bfe6490) [ CSSD]2013-05-23 16:12:29.591 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6490 (RPC#1257) state 6, flags 0x202 [ CSSD]2013-05-23 16:12:29.591 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6490, ret 0, client (nil) [ CSSD]2013-05-23 16:12:29.591 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1257 [ CSSD]2013-05-23 16:12:30.791 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:30.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:30.804 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:31.123 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:31.123 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:31.445 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:31.446 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:12:31.447 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:31.447 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6b70) proc(0x7f431c137b90) pid(4176) version 11:1:1:4 [ CSSD]2013-05-23 16:12:31.453 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:31.453 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:31.453 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:12:31.453 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:31.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1258 (0x7f431bfe6510) [ CSSD]2013-05-23 16:12:31.454 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6510 (RPC#1258) tag(4ea002c) sent to node 1 [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ea002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014419, gin 1, member 0 [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6510 (RPC#1258) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6510, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:31.454 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:31.455 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1258 [ CSSD]2013-05-23 16:12:31.462 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:31.462 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:31.462 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:31.462 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1259 (0x7f431bfe6590) [ CSSD]2013-05-23 16:12:31.462 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6590 (RPC#1259) tag(4eb002c) sent to node 1 [ CSSD]2013-05-23 16:12:31.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04eb002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:31.463 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:31.463 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6590 (RPC#1259) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:31.463 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6590, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:31.463 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1259 [ CSSD]2013-05-23 16:12:31.463 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:31.666 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:31.666 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c0e6b70) skgpid ospid 4176 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:32.809 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:32.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:32.828 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:34.815 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:34.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:34.840 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:36.126 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:36.127 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:36.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:36.845 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:36.845 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:38.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:38.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:38.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:39.614 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:12:39.614 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1260 (0x7f431bfe6610) [ CSSD]2013-05-23 16:12:39.615 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6610 (RPC#1260) tag(4ec002c) sent to node 1 [ CSSD]2013-05-23 16:12:39.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:39.615 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1260 (0x7f431bfe6610) [ CSSD]2013-05-23 16:12:39.615 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6610 (RPC#1260) state 6, flags 0x202 [ CSSD]2013-05-23 16:12:39.615 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6610, ret 0, client (nil) [ CSSD]2013-05-23 16:12:39.615 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1260 [ CSSD]2013-05-23 16:12:40.861 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:40.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:40.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:41.130 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:41.130 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:42.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:42.017 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014420, gin 1, member 0 [ CSSD]2013-05-23 16:12:42.017 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:12:42.017 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:12:42.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:42.023 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:12:42.023 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:12:42.870 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:42.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:42.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:44.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:44.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:44.922 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:45.133 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:45.133 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:12:46.703 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:12:46.703 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:12:46.704 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:12:46.704 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c137b90) pid(4400) version 11:1:1:4 [ CSSD]2013-05-23 16:12:46.710 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:12:46.710 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:46.710 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6b70, requested num -1 [ CSSD]2013-05-23 16:12:46.710 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:46.710 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1261 (0x7f431bfe6690) [ CSSD]2013-05-23 16:12:46.710 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6690 (RPC#1261) tag(4ed002c) sent to node 1 [ CSSD]2013-05-23 16:12:46.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ed002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:46.710 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014421, gin 1, member 0 [ CSSD]2013-05-23 16:12:46.710 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:12:46.711 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:12:46.711 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:12:46.711 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6690 (RPC#1261) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:46.711 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6690, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:46.711 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:12:46.711 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1261 [ CSSD]2013-05-23 16:12:46.717 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:12:46.717 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:46.717 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:12:46.717 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1262 (0x7f431bfe6710) [ CSSD]2013-05-23 16:12:46.717 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6710 (RPC#1262) tag(4ee002c) sent to node 1 [ CSSD]2013-05-23 16:12:46.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04ee002c) incarnation(174631051) [ CSSD]2013-05-23 16:12:46.717 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:12:46.717 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6710 (RPC#1262) state 6, flags 0x100 [ CSSD]2013-05-23 16:12:46.717 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6710, ret 0, client 0x7f431c12ba00 member 0x7f431c13be80 [ CSSD]2013-05-23 16:12:46.718 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1262 [ CSSD]2013-05-23 16:12:46.718 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:12:46.902 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:46.920 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:12:46.920 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c0e6e20) skgpid ospid 4400 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:46.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:46.927 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:48.955 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:48.959 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:48.959 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:49.004 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22eee80) [ CSSD]2013-05-23 16:12:49.004 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22eee80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:12:49.004 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22eee80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:12:49.004 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:12:49.004 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc670 [ CSSD]2013-05-23 16:12:49.004 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc670) con(0x2303710) skgpid 31286 ospid 31286 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:49.008 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22dacd0) [ CSSD]2013-05-23 16:12:49.008 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22dacd0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:12:49.008 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22dacd0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:12:49.008 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:12:49.008 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:12:49.008 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2312880) skgpid 31288 ospid 31288 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:12:49.639 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:12:49.640 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1263 (0x7f431bfe6790) [ CSSD]2013-05-23 16:12:49.640 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6790 (RPC#1263) tag(4ef002c) sent to node 1 [ CSSD]2013-05-23 16:12:49.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:49.640 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1263 (0x7f431bfe6790) [ CSSD]2013-05-23 16:12:49.640 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6790 (RPC#1263) state 6, flags 0x202 [ CSSD]2013-05-23 16:12:49.640 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6790, ret 0, client (nil) [ CSSD]2013-05-23 16:12:49.640 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1263 [ CSSD]2013-05-23 16:12:50.136 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:50.137 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:50.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:50.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:50.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:52.972 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:52.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:52.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:54.980 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:55.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:55.023 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:55.140 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:12:55.140 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:12:56.986 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:57.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:57.029 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:57.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:57.267 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014422, gin 1, member 0 [ CSSD]2013-05-23 16:12:57.267 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:12:57.267 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:12:57.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:57.274 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:12:57.274 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:12:58.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:12:59.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:59.040 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:12:59.663 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:12:59.664 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1264 (0x7f431bfe6810) [ CSSD]2013-05-23 16:12:59.664 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6810 (RPC#1264) tag(4f0002c) sent to node 1 [ CSSD]2013-05-23 16:12:59.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:12:59.665 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1264 (0x7f431bfe6810) [ CSSD]2013-05-23 16:12:59.665 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6810 (RPC#1264) state 6, flags 0x202 [ CSSD]2013-05-23 16:12:59.665 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6810, ret 0, client (nil) [ CSSD]2013-05-23 16:12:59.665 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1264 [ CSSD]2013-05-23 16:13:00.144 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:00.144 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:01.021 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:01.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:01.045 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:01.967 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:13:01.968 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:13:01.969 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:13:01.969 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c137b90) pid(4559) version 11:1:1:4 [ CSSD]2013-05-23 16:13:01.975 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:13:01.975 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:13:01.975 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:13:01.975 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:01.975 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1265 (0x7f431bfe6890) [ CSSD]2013-05-23 16:13:01.975 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6890 (RPC#1265) tag(4f1002c) sent to node 1 [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04f1002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014423, gin 1, member 0 [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bc30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6890 (RPC#1265) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6890, ret 0, client 0x7f431c12ba00 member 0x7f431c0955a0 [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:13:01.976 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1265 [ CSSD]2013-05-23 16:13:01.982 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:13:01.982 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:01.982 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0955a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:01.982 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1266 (0x7f431bfe6910) [ CSSD]2013-05-23 16:13:01.982 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6910 (RPC#1266) tag(4f2002c) sent to node 1 [ CSSD]2013-05-23 16:13:01.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04f2002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:01.983 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:01.983 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6910 (RPC#1266) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:01.983 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6910, ret 0, client 0x7f431c12ba00 member 0x7f431c0955a0 [ CSSD]2013-05-23 16:13:01.983 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1266 [ CSSD]2013-05-23 16:13:01.983 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0955a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:13:02.185 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:13:02.185 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c0e6e20) skgpid ospid 4559 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:13:03.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:03.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:03.072 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:05.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:05.103 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:05.103 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:05.148 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:05.148 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:07.085 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:07.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:07.109 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:09.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:09.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:09.176 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:09.689 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:13:09.689 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1267 (0x7f431bfe6990) [ CSSD]2013-05-23 16:13:09.690 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6990 (RPC#1267) tag(4f3002c) sent to node 1 [ CSSD]2013-05-23 16:13:09.690 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:09.690 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1267 (0x7f431bfe6990) [ CSSD]2013-05-23 16:13:09.690 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6990 (RPC#1267) state 6, flags 0x202 [ CSSD]2013-05-23 16:13:09.690 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6990, ret 0, client (nil) [ CSSD]2013-05-23 16:13:09.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1267 [ CSSD]2013-05-23 16:13:10.151 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:10.151 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:11.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:11.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:11.183 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:12.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:12.527 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014424, gin 1, member 0 [ CSSD]2013-05-23 16:13:12.527 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:13:12.527 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:13:12.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:12.533 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:13:12.533 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:13:13.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:13.133 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:13.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:13.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:13.456 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014425, gin 1, member 0 [ CSSD]2013-05-23 16:13:13.456 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:13:13.456 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:13:13.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:13.462 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:13:13.462 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:13:15.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:15.143 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:15.155 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:15.155 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:15.239 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:17.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:17.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:17.223 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b90) allocated [ CSSD]2013-05-23 16:13:17.224 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:13:17.224 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b90 - 1,2,3 [ CSSD]2013-05-23 16:13:17.224 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137b90) pid(4699) version 11:1:1:4 [ CSSD]2013-05-23 16:13:17.230 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c137b90), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:13:17.231 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:13:17.231 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:13:17.231 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:17.231 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1268 (0x7f431bfe6a10) [ CSSD]2013-05-23 16:13:17.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6a10 (RPC#1268) tag(4f4002c) sent to node 1 [ CSSD]2013-05-23 16:13:17.231 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04f4002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:17.231 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014426, gin 1, member 0 [ CSSD]2013-05-23 16:13:17.231 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13bb70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:13:17.231 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:13:17.232 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:13:17.232 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6a10 (RPC#1268) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:17.232 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6a10, ret 0, client 0x7f431c12ba00 member 0x7f431c13be70 [ CSSD]2013-05-23 16:13:17.232 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:13:17.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1268 [ CSSD]2013-05-23 16:13:17.240 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:13:17.240 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:17.240 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13be70) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:17.240 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1269 (0x7f431bfe6a90) [ CSSD]2013-05-23 16:13:17.240 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6a90 (RPC#1269) tag(4f5002c) sent to node 1 [ CSSD]2013-05-23 16:13:17.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04f5002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:17.241 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:17.241 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6a90 (RPC#1269) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:17.241 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6a90, ret 0, client 0x7f431c12ba00 member 0x7f431c13be70 [ CSSD]2013-05-23 16:13:17.241 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1269 [ CSSD]2013-05-23 16:13:17.241 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13be70) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:13:17.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:17.261 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13a9d0) allocated [ CSSD]2013-05-23 16:13:17.262 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:13:17.262 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13a9d0 - 1,2,3 [ CSSD]2013-05-23 16:13:17.262 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c144780) proc(0x7f431c13a9d0) pid(4710) version 11:1:1:4 [ CSSD]2013-05-23 16:13:17.268 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13a9d0), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:13:17.268 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:13:17.268 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:13:17.268 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:17.268 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1270 (0x7f431bfe6b10) [ CSSD]2013-05-23 16:13:17.268 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6b10 (RPC#1270) tag(4f6002c) sent to node 1 [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04f6002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014427, gin 1, member 0 [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1366f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6b10 (RPC#1270) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:17.268 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6b10, ret 0, client 0x7f431c12ba00 member 0x7f431c13a8b0 [ CSSD]2013-05-23 16:13:17.269 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:13:17.269 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1270 [ CSSD]2013-05-23 16:13:17.274 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:13:17.274 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:17.274 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13a8b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:17.274 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1271 (0x7f431bfe6b90) [ CSSD]2013-05-23 16:13:17.274 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6b90 (RPC#1271) tag(4f7002c) sent to node 1 [ CSSD]2013-05-23 16:13:17.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04f7002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:17.274 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:17.274 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6b90 (RPC#1271) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:17.274 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6b90, ret 0, client 0x7f431c12ba00 member 0x7f431c13a8b0 [ CSSD]2013-05-23 16:13:17.275 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1271 [ CSSD]2013-05-23 16:13:17.275 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13a8b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:13:17.444 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b90 [ CSSD]2013-05-23 16:13:17.444 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b90) con(0x7f431c11ab50) skgpid ospid 4699 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:13:17.477 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13a9d0 [ CSSD]2013-05-23 16:13:17.477 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13a9d0) con(0x7f431c144780) skgpid ospid 4710 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:13:19.158 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:19.158 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:13:19.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:19.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:19.256 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:19.713 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:13:19.713 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1272 (0x7f431bfe6c10) [ CSSD]2013-05-23 16:13:19.714 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6c10 (RPC#1272) tag(4f8002c) sent to node 1 [ CSSD]2013-05-23 16:13:19.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:19.714 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1272 (0x7f431bfe6c10) [ CSSD]2013-05-23 16:13:19.714 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6c10 (RPC#1272) state 6, flags 0x202 [ CSSD]2013-05-23 16:13:19.714 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6c10, ret 0, client (nil) [ CSSD]2013-05-23 16:13:19.714 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1272 [ CSSD]2013-05-23 16:13:21.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:21.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:21.263 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:23.213 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:23.213 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:23.274 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:24.162 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:24.162 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:25.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:25.223 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:25.280 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:27.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:27.249 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:27.286 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:27.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:27.790 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014428, gin 1, member 0 [ CSSD]2013-05-23 16:13:27.790 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:13:27.790 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:13:27.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:27.830 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:13:27.830 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:13:29.165 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:29.166 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:29.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:29.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:29.302 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:29.737 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:13:29.737 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1273 (0x7f431bfe6c90) [ CSSD]2013-05-23 16:13:29.738 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6c90 (RPC#1273) tag(4f9002c) sent to node 1 [ CSSD]2013-05-23 16:13:29.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:29.738 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1273 (0x7f431bfe6c90) [ CSSD]2013-05-23 16:13:29.738 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6c90 (RPC#1273) state 6, flags 0x202 [ CSSD]2013-05-23 16:13:29.738 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6c90, ret 0, client (nil) [ CSSD]2013-05-23 16:13:29.738 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1273 [ CSSD]2013-05-23 16:13:31.304 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:31.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:31.328 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:32.483 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 16:13:32.484 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:13:32.485 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 16:13:32.485 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22fc9c0) pid(4854) version 11:1:1:4 [ CSSD]2013-05-23 16:13:32.491 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22fc9c0), client(1/0x2312950) [ CSSD]2013-05-23 16:13:32.491 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:13:32.491 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 16:13:32.491 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:32.491 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1274 (0x7f431bfe6d10) [ CSSD]2013-05-23 16:13:32.491 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6d10 (RPC#1274) tag(4fa002c) sent to node 1 [ CSSD]2013-05-23 16:13:32.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04fa002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:32.491 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014429, gin 1, member 0 [ CSSD]2013-05-23 16:13:32.491 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:13:32.492 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:13:32.492 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:13:32.492 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6d10 (RPC#1274) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:32.492 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6d10, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:13:32.492 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 16:13:32.492 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1274 [ CSSD]2013-05-23 16:13:32.498 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:13:32.498 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:32.498 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:32.498 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1275 (0x7f431bfe6d90) [ CSSD]2013-05-23 16:13:32.498 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6d90 (RPC#1275) tag(4fb002c) sent to node 1 [ CSSD]2013-05-23 16:13:32.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04fb002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:32.499 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:32.499 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6d90 (RPC#1275) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:32.499 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6d90, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:13:32.499 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1275 [ CSSD]2013-05-23 16:13:32.499 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:13:32.701 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 16:13:32.701 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x205ecb0) skgpid ospid 4854 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:13:33.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:33.334 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:33.354 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:34.169 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:34.169 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:35.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:35.343 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:35.379 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:37.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:37.348 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:37.471 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:37.760 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:37.760 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014430, gin 1, member 0 [ CSSD]2013-05-23 16:13:37.760 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:13:37.760 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:13:37.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:37.767 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:13:37.767 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:13:37.892 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:13:37.892 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:13:37.893 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:13:37.894 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:13:37.894 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:13:37.895 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:13:39.173 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:39.173 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:39.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:39.402 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:39.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:39.767 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:13:39.767 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1276 (0x7f431bfe6e10) [ CSSD]2013-05-23 16:13:39.768 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6e10 (RPC#1276) tag(4fc002c) sent to node 1 [ CSSD]2013-05-23 16:13:39.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:39.768 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1276 (0x7f431bfe6e10) [ CSSD]2013-05-23 16:13:39.768 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6e10 (RPC#1276) state 6, flags 0x202 [ CSSD]2013-05-23 16:13:39.768 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6e10, ret 0, client (nil) [ CSSD]2013-05-23 16:13:39.768 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1276 [ CSSD]2013-05-23 16:13:40.007 [1191196992] >TRACE: clssgmAllocProc: (0x231d6c0) allocated [ CSSD]2013-05-23 16:13:40.008 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:13:40.008 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231d6c0 - 1,2,3 [ CSSD]2013-05-23 16:13:40.008 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x231d6c0) pid(4461/4461) version 11:1:1:4 [ CSSD]2013-05-23 16:13:40.024 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231d6c0), client(1/0x2312950) [ CSSD]2013-05-23 16:13:40.024 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:13:40.024 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:13:40.024 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:13:40.463 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231d6c0), client(2/0x205ed60) [ CSSD]2013-05-23 16:13:40.463 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x205ed60) [ CSSD]2013-05-23 16:13:40.463 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:13:40.463 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:13:41.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:41.407 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:41.523 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:43.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:43.034 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014431, gin 1, member 0 [ CSSD]2013-05-23 16:13:43.034 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:13:43.034 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:13:43.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:43.039 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:13:43.039 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7750) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:13:43.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:43.411 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:43.555 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:44.177 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:44.177 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:45.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:45.424 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:45.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:47.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:47.449 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:47.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:47.737 [1191196992] >TRACE: clssgmAllocProc: (0x22eec70) allocated [ CSSD]2013-05-23 16:13:47.738 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:13:47.738 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22eec70 - 1,2,3 [ CSSD]2013-05-23 16:13:47.738 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7870) proc(0x22eec70) pid(4999) version 11:1:1:4 [ CSSD]2013-05-23 16:13:47.744 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22eec70), client(1/0x22f7ce0) [ CSSD]2013-05-23 16:13:47.744 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:13:47.744 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f7ce0 with con 0x22f8850, requested num -1 [ CSSD]2013-05-23 16:13:47.744 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:47.744 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1277 (0x7f431bfe6e90) [ CSSD]2013-05-23 16:13:47.744 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6e90 (RPC#1277) tag(4fd002c) sent to node 1 [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04fd002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014432, gin 1, member 0 [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6e90 (RPC#1277) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe6e90, ret 0, client 0x22f7ce0 member 0x22ff740 [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f7ce0 [ CSSD]2013-05-23 16:13:47.745 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1277 [ CSSD]2013-05-23 16:13:47.752 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:13:47.752 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f7ce0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:47.752 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ff740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:13:47.752 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1278 (0x7f431bfe6f10) [ CSSD]2013-05-23 16:13:47.752 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6f10 (RPC#1278) tag(4fe002c) sent to node 1 [ CSSD]2013-05-23 16:13:47.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04fe002c) incarnation(174631051) [ CSSD]2013-05-23 16:13:47.752 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:13:47.752 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6f10 (RPC#1278) state 6, flags 0x100 [ CSSD]2013-05-23 16:13:47.752 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe6f10, ret 0, client 0x22f7ce0 member 0x22ff740 [ CSSD]2013-05-23 16:13:47.753 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1278 [ CSSD]2013-05-23 16:13:47.753 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ff740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:13:47.955 [1191196992] >TRACE: clssgmDeadProc: proc 0x22eec70 [ CSSD]2013-05-23 16:13:47.955 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22eec70) con(0x22d7870) skgpid ospid 4999 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:13:49.181 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:49.181 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:49.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:49.453 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:49.598 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:49.792 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:13:49.792 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1279 (0x7f431bfe6f90) [ CSSD]2013-05-23 16:13:49.793 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe6f90 (RPC#1279) tag(4ff002c) sent to node 1 [ CSSD]2013-05-23 16:13:49.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:49.793 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1279 (0x7f431bfe6f90) [ CSSD]2013-05-23 16:13:49.793 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe6f90 (RPC#1279) state 6, flags 0x202 [ CSSD]2013-05-23 16:13:49.793 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe6f90, ret 0, client (nil) [ CSSD]2013-05-23 16:13:49.793 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1279 [ CSSD]2013-05-23 16:13:50.518 [1191196992] >TRACE: clssgmAllocProc: (0x22ff7a0) allocated [ CSSD]2013-05-23 16:13:50.519 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:13:50.520 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ff7a0 - 1,2,3 [ CSSD]2013-05-23 16:13:50.520 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2303710) proc(0x22ff7a0) pid(5062/5062) version 11:1:1:4 [ CSSD]2013-05-23 16:13:50.535 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ff7a0), client(1/0x22f7ce0) [ CSSD]2013-05-23 16:13:50.535 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:13:50.535 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:13:50.535 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:13:51.459 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:51.459 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:51.623 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:53.184 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:53.184 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:13:53.325 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c144f40) [ CSSD]2013-05-23 16:13:53.325 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c144f40), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:13:53.325 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c144f40), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:13:53.325 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:13:53.338 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c144540) [ CSSD]2013-05-23 16:13:53.338 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c144540), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:13:53.338 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c144540), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:13:53.338 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:13:53.338 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c143f20 [ CSSD]2013-05-23 16:13:53.338 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c143f20) con(0x7f431c141ca0) skgpid 30357 ospid 30357 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:13:53.484 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:53.484 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:53.674 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:53.848 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:13:53.849 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:13:53.850 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:13:53.850 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(5097/5097) version 11:1:1:4 [ CSSD]2013-05-23 16:13:53.866 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x231ed60), client(1/0x23029d0) [ CSSD]2013-05-23 16:13:53.866 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23029d0) [ CSSD]2013-05-23 16:13:53.866 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:13:53.866 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:13:53.889 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x231ed60), client(2/0x22f1c80) [ CSSD]2013-05-23 16:13:53.889 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f1c80) [ CSSD]2013-05-23 16:13:53.889 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:13:53.889 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:13:55.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:55.534 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:55.681 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:57.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:57.540 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:57.693 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:58.187 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:13:58.188 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:13:58.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:58.282 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014433, gin 1, member 0 [ CSSD]2013-05-23 16:13:58.282 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2317970/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:13:58.282 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:13:58.289 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:58.289 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:13:58.289 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1a70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:13:59.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:59.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:13:59.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:13:59.820 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:13:59.820 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1280 (0x7f431bfe7010) [ CSSD]2013-05-23 16:13:59.821 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7010 (RPC#1280) tag(500002c) sent to node 1 [ CSSD]2013-05-23 16:13:59.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:13:59.821 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1280 (0x7f431bfe7010) [ CSSD]2013-05-23 16:13:59.821 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7010 (RPC#1280) state 6, flags 0x202 [ CSSD]2013-05-23 16:13:59.821 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7010, ret 0, client (nil) [ CSSD]2013-05-23 16:13:59.821 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1280 [ CSSD]2013-05-23 16:14:01.559 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:01.559 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:01.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:02.988 [1191196992] >TRACE: clssgmAllocProc: (0x22f4b40) allocated [ CSSD]2013-05-23 16:14:02.989 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:14:02.990 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4b40 - 1,2,3 [ CSSD]2013-05-23 16:14:02.990 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302b40) proc(0x22f4b40) pid(5441) version 11:1:1:4 [ CSSD]2013-05-23 16:14:02.996 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22f4b40), client(1/0x2317dd0) [ CSSD]2013-05-23 16:14:02.996 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317dd0) [ CSSD]2013-05-23 16:14:02.996 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317dd0 with con 0x22f4e10, requested num -1 [ CSSD]2013-05-23 16:14:02.996 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:02.996 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1281 (0x7f431bfe7090) [ CSSD]2013-05-23 16:14:02.996 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7090 (RPC#1281) tag(501002c) sent to node 1 [ CSSD]2013-05-23 16:14:02.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0501002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:02.996 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014434, gin 1, member 0 [ CSSD]2013-05-23 16:14:02.996 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317f40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:14:02.997 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:14:02.997 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:14:02.997 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7090 (RPC#1281) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:02.997 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7090, ret 0, client 0x2317dd0 member 0x22f4a20 [ CSSD]2013-05-23 16:14:02.997 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317dd0 [ CSSD]2013-05-23 16:14:02.997 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1281 [ CSSD]2013-05-23 16:14:03.003 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317dd0) [ CSSD]2013-05-23 16:14:03.003 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317dd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:03.003 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4a20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:03.003 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1282 (0x7f431bfe7110) [ CSSD]2013-05-23 16:14:03.003 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7110 (RPC#1282) tag(502002c) sent to node 1 [ CSSD]2013-05-23 16:14:03.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0502002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:03.003 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:03.003 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7110 (RPC#1282) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:03.003 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7110, ret 0, client 0x2317dd0 member 0x22f4a20 [ CSSD]2013-05-23 16:14:03.004 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1282 [ CSSD]2013-05-23 16:14:03.004 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4a20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:14:03.191 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:03.191 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:03.206 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4b40 [ CSSD]2013-05-23 16:14:03.206 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4b40) con(0x2302b40) skgpid ospid 5441 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:14:03.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:03.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:03.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:05.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:05.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:05.712 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:07.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:07.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:07.764 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:08.195 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:08.195 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:09.733 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:09.733 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:09.789 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:09.836 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:14:09.836 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f7ce0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:14:09.836 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f7ce0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:14:09.836 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:14:09.836 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ff7a0 [ CSSD]2013-05-23 16:14:09.836 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ff7a0) con(0x2303710) skgpid 5062 ospid 5062 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:14:09.845 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:14:09.845 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1283 (0x7f431bfe7190) [ CSSD]2013-05-23 16:14:09.846 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7190 (RPC#1283) tag(503002c) sent to node 1 [ CSSD]2013-05-23 16:14:09.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:09.846 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1283 (0x7f431bfe7190) [ CSSD]2013-05-23 16:14:09.846 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7190 (RPC#1283) state 6, flags 0x202 [ CSSD]2013-05-23 16:14:09.846 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7190, ret 0, client (nil) [ CSSD]2013-05-23 16:14:09.846 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1283 [ CSSD]2013-05-23 16:14:11.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:11.738 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:11.796 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:13.199 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:13.199 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:13.542 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:13.542 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014435, gin 1, member 0 [ CSSD]2013-05-23 16:14:13.542 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:14:13.542 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:14:13.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:13.549 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:14:13.549 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:14:13.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:13.732 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014436, gin 1, member 0 [ CSSD]2013-05-23 16:14:13.732 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:14:13.732 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:14:13.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:13.739 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:14:13.739 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:14:13.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:13.744 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:13.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:15.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:15.756 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:15.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:17.522 [1191196992] >TRACE: clssgmAllocProc: (0x22ff670) allocated [ CSSD]2013-05-23 16:14:17.524 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:14:17.525 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ff670 - 1,2,3 [ CSSD]2013-05-23 16:14:17.525 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1d20) proc(0x22ff670) pid(5577) version 11:1:1:4 [ CSSD]2013-05-23 16:14:17.531 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ff670), client(1/0x22f7ce0) [ CSSD]2013-05-23 16:14:17.531 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:14:17.531 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f7ce0 with con 0x22f87f0, requested num -1 [ CSSD]2013-05-23 16:14:17.531 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:17.531 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1284 (0x7f431bfe7210) [ CSSD]2013-05-23 16:14:17.531 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7210 (RPC#1284) tag(504002c) sent to node 1 [ CSSD]2013-05-23 16:14:17.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0504002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:17.531 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014437, gin 1, member 0 [ CSSD]2013-05-23 16:14:17.531 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f7f10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:14:17.532 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:14:17.532 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:14:17.532 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7210 (RPC#1284) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:17.532 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7210, ret 0, client 0x22f7ce0 member 0x2317b80 [ CSSD]2013-05-23 16:14:17.532 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f7ce0 [ CSSD]2013-05-23 16:14:17.532 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1284 [ CSSD]2013-05-23 16:14:17.538 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:14:17.538 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f7ce0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:17.538 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2317b80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:17.538 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1285 (0x7f431bfe7290) [ CSSD]2013-05-23 16:14:17.538 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7290 (RPC#1285) tag(505002c) sent to node 1 [ CSSD]2013-05-23 16:14:17.538 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0505002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:17.538 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:17.538 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7290 (RPC#1285) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:17.538 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7290, ret 0, client 0x22f7ce0 member 0x2317b80 [ CSSD]2013-05-23 16:14:17.539 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1285 [ CSSD]2013-05-23 16:14:17.539 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2317b80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:14:17.741 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ff670 [ CSSD]2013-05-23 16:14:17.741 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ff670) con(0x22f1d20) skgpid ospid 5577 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:14:17.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:17.762 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:17.864 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:18.203 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:18.203 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:18.247 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1297d0) allocated [ CSSD]2013-05-23 16:14:18.248 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:14:18.249 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1297d0 - 1,2,3 [ CSSD]2013-05-23 16:14:18.249 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c1297d0) pid(5592) version 11:1:1:4 [ CSSD]2013-05-23 16:14:18.254 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1297d0), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:14:18.254 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:14:18.254 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:14:18.254 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:18.254 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1286 (0x7f431bfe7310) [ CSSD]2013-05-23 16:14:18.254 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7310 (RPC#1286) tag(506002c) sent to node 1 [ CSSD]2013-05-23 16:14:18.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0506002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:18.254 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014438, gin 1, member 0 [ CSSD]2013-05-23 16:14:18.254 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:14:18.255 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:14:18.255 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:14:18.255 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7310 (RPC#1286) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:18.255 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7310, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:14:18.255 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:14:18.255 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1286 [ CSSD]2013-05-23 16:14:18.261 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:14:18.261 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:18.261 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:18.261 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1287 (0x7f431bfe7390) [ CSSD]2013-05-23 16:14:18.261 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7390 (RPC#1287) tag(507002c) sent to node 1 [ CSSD]2013-05-23 16:14:18.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0507002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:18.262 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:18.262 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7390 (RPC#1287) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:18.262 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7390, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:14:18.262 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1287 [ CSSD]2013-05-23 16:14:18.262 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:14:18.464 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1297d0 [ CSSD]2013-05-23 16:14:18.464 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1297d0) con(0x7f431c0e6e20) skgpid ospid 5592 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:14:19.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:19.815 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:19.870 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:14:19.870 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1288 (0x7f431bfe7410) [ CSSD]2013-05-23 16:14:19.870 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7410 (RPC#1288) tag(508002c) sent to node 1 [ CSSD]2013-05-23 16:14:19.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:19.871 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1288 (0x7f431bfe7410) [ CSSD]2013-05-23 16:14:19.871 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7410 (RPC#1288) state 6, flags 0x202 [ CSSD]2013-05-23 16:14:19.871 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7410, ret 0, client (nil) [ CSSD]2013-05-23 16:14:19.871 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1288 [ CSSD]2013-05-23 16:14:19.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:21.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:21.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:21.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:23.206 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:23.206 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:23.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:23.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:23.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:24.766 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:14:24.766 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:14:24.767 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:14:24.768 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:14:24.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:24.849 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014439, gin 1, member 0 [ CSSD]2013-05-23 16:14:24.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f75a0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:14:24.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:14:24.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:24.856 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:14:24.856 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:14:25.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:25.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:25.913 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:27.209 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:27.209 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:14:27.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:27.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:27.965 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:28.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:28.787 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014440, gin 1, member 0 [ CSSD]2013-05-23 16:14:28.787 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:14:28.787 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:14:28.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:28.793 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:14:28.793 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:14:29.895 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:14:29.896 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1289 (0x7f431bfe7490) [ CSSD]2013-05-23 16:14:29.896 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7490 (RPC#1289) tag(509002c) sent to node 1 [ CSSD]2013-05-23 16:14:29.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:29.897 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1289 (0x7f431bfe7490) [ CSSD]2013-05-23 16:14:29.897 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7490 (RPC#1289) state 6, flags 0x202 [ CSSD]2013-05-23 16:14:29.897 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7490, ret 0, client (nil) [ CSSD]2013-05-23 16:14:29.897 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1289 [ CSSD]2013-05-23 16:14:29.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:29.915 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:29.971 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:31.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:31.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:31.977 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:32.213 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:32.213 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:33.513 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1297d0) allocated [ CSSD]2013-05-23 16:14:33.514 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:14:33.515 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1297d0 - 1,2,3 [ CSSD]2013-05-23 16:14:33.515 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c1297d0) pid(5754) version 11:1:1:4 [ CSSD]2013-05-23 16:14:33.520 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1297d0), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:14:33.520 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:14:33.520 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:14:33.520 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:33.520 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1290 (0x7f431bfe7510) [ CSSD]2013-05-23 16:14:33.521 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7510 (RPC#1290) tag(50a002c) sent to node 1 [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(050a002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014441, gin 1, member 0 [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7510 (RPC#1290) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7510, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:14:33.521 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1290 [ CSSD]2013-05-23 16:14:33.528 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:14:33.528 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:33.528 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:33.528 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1291 (0x7f431bfe7590) [ CSSD]2013-05-23 16:14:33.528 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7590 (RPC#1291) tag(50b002c) sent to node 1 [ CSSD]2013-05-23 16:14:33.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(050b002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:33.528 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:33.528 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7590 (RPC#1291) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:33.528 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7590, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:14:33.528 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1291 [ CSSD]2013-05-23 16:14:33.528 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:14:33.731 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1297d0 [ CSSD]2013-05-23 16:14:33.731 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1297d0) con(0x7f431c11ab50) skgpid ospid 5754 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:14:33.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:33.933 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:34.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:35.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:35.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:36.033 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:37.217 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:37.217 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:37.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:37.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:38.065 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:39.920 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:14:39.920 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1292 (0x7f431bfe7610) [ CSSD]2013-05-23 16:14:39.920 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7610 (RPC#1292) tag(50c002c) sent to node 1 [ CSSD]2013-05-23 16:14:39.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:39.921 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1292 (0x7f431bfe7610) [ CSSD]2013-05-23 16:14:39.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7610 (RPC#1292) state 6, flags 0x202 [ CSSD]2013-05-23 16:14:39.921 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7610, ret 0, client (nil) [ CSSD]2013-05-23 16:14:39.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1292 [ CSSD]2013-05-23 16:14:40.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:40.016 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:40.072 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:41.220 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:41.220 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:14:42.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:42.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:42.078 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:44.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:44.034 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:44.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:44.046 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014442, gin 1, member 0 [ CSSD]2013-05-23 16:14:44.046 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:14:44.046 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:14:44.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:44.053 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:14:44.053 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:14:44.088 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:46.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:46.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:46.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:46.224 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:46.224 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:48.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:48.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:48.166 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:48.767 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1297d0) allocated [ CSSD]2013-05-23 16:14:48.768 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:14:48.769 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1297d0 - 1,2,3 [ CSSD]2013-05-23 16:14:48.769 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c1297d0) pid(5910) version 11:1:1:4 [ CSSD]2013-05-23 16:14:48.775 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1297d0), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:14:48.775 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:14:48.775 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6b70, requested num -1 [ CSSD]2013-05-23 16:14:48.775 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:48.775 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1293 (0x7f431bfe7690) [ CSSD]2013-05-23 16:14:48.775 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7690 (RPC#1293) tag(50d002c) sent to node 1 [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(050d002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014443, gin 1, member 0 [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7690 (RPC#1293) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7690, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:14:48.776 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1293 [ CSSD]2013-05-23 16:14:48.782 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:14:48.782 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:48.782 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:14:48.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1294 (0x7f431bfe7710) [ CSSD]2013-05-23 16:14:48.782 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7710 (RPC#1294) tag(50e002c) sent to node 1 [ CSSD]2013-05-23 16:14:48.782 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(050e002c) incarnation(174631051) [ CSSD]2013-05-23 16:14:48.782 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:14:48.782 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7710 (RPC#1294) state 6, flags 0x100 [ CSSD]2013-05-23 16:14:48.782 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7710, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:14:48.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1294 [ CSSD]2013-05-23 16:14:48.783 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:14:48.985 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1297d0 [ CSSD]2013-05-23 16:14:48.985 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1297d0) con(0x7f431c11ab50) skgpid ospid 5910 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:14:49.943 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:14:49.943 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1295 (0x7f431bfe7790) [ CSSD]2013-05-23 16:14:49.943 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7790 (RPC#1295) tag(50f002c) sent to node 1 [ CSSD]2013-05-23 16:14:49.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:49.944 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1295 (0x7f431bfe7790) [ CSSD]2013-05-23 16:14:49.944 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7790 (RPC#1295) state 6, flags 0x202 [ CSSD]2013-05-23 16:14:49.944 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7790, ret 0, client (nil) [ CSSD]2013-05-23 16:14:49.944 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1295 [ CSSD]2013-05-23 16:14:50.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:50.116 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:50.172 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:51.228 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:51.228 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:52.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:52.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:52.179 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:54.134 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:54.134 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:54.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:56.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:56.139 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:56.195 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:56.231 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:14:56.232 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:14:58.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:58.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:14:58.206 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:14:59.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:59.300 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014444, gin 1, member 0 [ CSSD]2013-05-23 16:14:59.300 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:14:59.300 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:14:59.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:59.306 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:14:59.307 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:14:59.967 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:14:59.968 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1296 (0x7f431bfe7810) [ CSSD]2013-05-23 16:14:59.968 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7810 (RPC#1296) tag(510002c) sent to node 1 [ CSSD]2013-05-23 16:14:59.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:14:59.968 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1296 (0x7f431bfe7810) [ CSSD]2013-05-23 16:14:59.968 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7810 (RPC#1296) state 6, flags 0x202 [ CSSD]2013-05-23 16:14:59.968 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7810, ret 0, client (nil) [ CSSD]2013-05-23 16:14:59.968 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1296 [ CSSD]2013-05-23 16:15:00.213 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:00.217 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:00.217 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:00.234 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:00.235 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:15:02.220 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:02.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:02.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:04.017 [1191196992] >TRACE: clssgmAllocProc: (0x22fc670) allocated [ CSSD]2013-05-23 16:15:04.018 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:15:04.018 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc670 - 1,2,3 [ CSSD]2013-05-23 16:15:04.018 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1d20) proc(0x22fc670) pid(6074) version 11:1:1:4 [ CSSD]2013-05-23 16:15:04.024 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22fc670), client(1/0x22eee80) [ CSSD]2013-05-23 16:15:04.024 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eee80) [ CSSD]2013-05-23 16:15:04.024 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eee80 with con 0x2302b40, requested num -1 [ CSSD]2013-05-23 16:15:04.024 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:04.024 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1297 (0x7f431bfe7890) [ CSSD]2013-05-23 16:15:04.025 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7890 (RPC#1297) tag(511002c) sent to node 1 [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0511002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014445, gin 1, member 0 [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317a40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7890 (RPC#1297) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7890, ret 0, client 0x22eee80 member 0x22ff830 [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eee80 [ CSSD]2013-05-23 16:15:04.025 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1297 [ CSSD]2013-05-23 16:15:04.031 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eee80) [ CSSD]2013-05-23 16:15:04.031 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eee80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:04.031 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ff830) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:04.031 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1298 (0x7f431bfe7910) [ CSSD]2013-05-23 16:15:04.032 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7910 (RPC#1298) tag(512002c) sent to node 1 [ CSSD]2013-05-23 16:15:04.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0512002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:04.032 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:04.032 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7910 (RPC#1298) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:04.032 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7910, ret 0, client 0x22eee80 member 0x22ff830 [ CSSD]2013-05-23 16:15:04.032 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1298 [ CSSD]2013-05-23 16:15:04.032 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ff830) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:15:04.235 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc670 [ CSSD]2013-05-23 16:15:04.235 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc670) con(0x22f1d20) skgpid ospid 6074 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:04.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:04.255 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:04.269 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:05.238 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:05.238 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:06.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:06.260 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:06.295 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:08.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:08.266 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:08.306 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:08.520 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1297d0) allocated [ CSSD]2013-05-23 16:15:08.521 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:15:08.521 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1297d0 - 1,2,3 [ CSSD]2013-05-23 16:15:08.521 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c1297d0) pid(6155) version 11:1:1:4 [ CSSD]2013-05-23 16:15:09.182 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1297d0 [ CSSD]2013-05-23 16:15:09.182 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1297d0) con(0x7f431c0e6e20) skgpid ospid 6155 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:09.992 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:15:09.993 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1299 (0x7f431bfe7990) [ CSSD]2013-05-23 16:15:09.993 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7990 (RPC#1299) tag(513002c) sent to node 1 [ CSSD]2013-05-23 16:15:09.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:09.993 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1299 (0x7f431bfe7990) [ CSSD]2013-05-23 16:15:09.994 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7990 (RPC#1299) state 6, flags 0x202 [ CSSD]2013-05-23 16:15:09.994 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7990, ret 0, client (nil) [ CSSD]2013-05-23 16:15:09.994 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1299 [ CSSD]2013-05-23 16:15:10.242 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:10.242 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:10.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:10.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:10.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:12.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:12.292 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:12.314 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:13.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:13.989 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014446, gin 1, member 0 [ CSSD]2013-05-23 16:15:13.990 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:15:13.990 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:15:14.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:14.030 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:15:14.030 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:15:14.245 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:14.245 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:15:14.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:14.325 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:14.361 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:14.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:14.549 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014447, gin 1, member 0 [ CSSD]2013-05-23 16:15:14.549 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:15:14.549 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:15:14.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:14.589 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:15:14.589 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:15:16.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:16.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:16.368 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:17.781 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1297d0) allocated [ CSSD]2013-05-23 16:15:17.783 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:15:17.783 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1297d0 - 1,2,3 [ CSSD]2013-05-23 16:15:17.783 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c1297d0) pid(6317) version 11:1:1:4 [ CSSD]2013-05-23 16:15:17.789 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1297d0), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:15:17.789 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:15:17.789 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:15:17.789 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:17.789 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1300 (0x7f431bfe7a10) [ CSSD]2013-05-23 16:15:17.789 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7a10 (RPC#1300) tag(514002c) sent to node 1 [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0514002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014448, gin 1, member 0 [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7a10 (RPC#1300) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7a10, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:15:17.790 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1300 [ CSSD]2013-05-23 16:15:17.797 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:15:17.797 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:17.797 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:17.797 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1301 (0x7f431bfe7a90) [ CSSD]2013-05-23 16:15:17.797 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7a90 (RPC#1301) tag(515002c) sent to node 1 [ CSSD]2013-05-23 16:15:17.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0515002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:17.797 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:17.797 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7a90 (RPC#1301) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:17.797 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7a90, ret 0, client 0x7f431c12ba00 member 0x7f431c13b990 [ CSSD]2013-05-23 16:15:17.797 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1301 [ CSSD]2013-05-23 16:15:17.797 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:15:18.000 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1297d0 [ CSSD]2013-05-23 16:15:18.000 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1297d0) con(0x7f431c0e6e20) skgpid ospid 6317 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:18.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:18.356 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:18.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:19.032 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:15:19.032 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:15:19.032 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:15:19.032 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:15:19.033 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147ab0 [ CSSD]2013-05-23 16:15:19.033 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147ab0) con(0x7f431c145810) skgpid 645 ospid 645 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:19.041 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:15:19.041 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:15:19.041 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:15:19.041 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:15:19.041 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:15:19.041 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c13fa10) skgpid 647 ospid 647 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:19.249 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:19.249 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:19.278 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 16:15:19.279 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:15:19.280 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 16:15:19.280 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c048500) pid(6347) version 11:1:1:4 [ CSSD]2013-05-23 16:15:19.286 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c048500), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:15:19.286 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:15:19.286 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:15:19.286 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:19.286 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1302 (0x7f431bfe7b10) [ CSSD]2013-05-23 16:15:19.286 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7b10 (RPC#1302) tag(516002c) sent to node 1 [ CSSD]2013-05-23 16:15:19.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0516002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:19.286 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014449, gin 1, member 0 [ CSSD]2013-05-23 16:15:19.286 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c147ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:15:19.287 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:15:19.287 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:15:19.287 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7b10 (RPC#1302) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:19.287 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7b10, ret 0, client 0x7f431c109f80 member 0x7f431c13ea10 [ CSSD]2013-05-23 16:15:19.287 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:15:19.287 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1302 [ CSSD]2013-05-23 16:15:19.292 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:15:19.292 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:19.292 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ea10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:19.292 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1303 (0x7f431bfe7b90) [ CSSD]2013-05-23 16:15:19.292 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7b90 (RPC#1303) tag(517002c) sent to node 1 [ CSSD]2013-05-23 16:15:19.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0517002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:19.293 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:19.293 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7b90 (RPC#1303) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:19.293 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7b90, ret 0, client 0x7f431c109f80 member 0x7f431c13ea10 [ CSSD]2013-05-23 16:15:19.293 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1303 [ CSSD]2013-05-23 16:15:19.293 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13ea10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:15:19.495 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 16:15:19.495 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c11ab50) skgpid ospid 6347 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:20.020 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:15:20.021 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1304 (0x7f431bfe7c10) [ CSSD]2013-05-23 16:15:20.021 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7c10 (RPC#1304) tag(518002c) sent to node 1 [ CSSD]2013-05-23 16:15:20.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:20.021 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1304 (0x7f431bfe7c10) [ CSSD]2013-05-23 16:15:20.021 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7c10 (RPC#1304) state 6, flags 0x202 [ CSSD]2013-05-23 16:15:20.021 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7c10, ret 0, client (nil) [ CSSD]2013-05-23 16:15:20.022 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1304 [ CSSD]2013-05-23 16:15:20.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:20.368 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:20.384 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:22.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:22.373 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:22.394 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:24.252 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:24.253 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:24.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:24.425 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:24.425 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:26.408 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:26.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:26.431 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:28.408 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:28.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:28.457 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:29.256 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:29.256 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:29.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:29.798 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014450, gin 1, member 0 [ CSSD]2013-05-23 16:15:29.798 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:15:29.798 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:15:29.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:29.804 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:15:29.804 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:15:30.043 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:15:30.043 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1305 (0x7f431bfe7c90) [ CSSD]2013-05-23 16:15:30.043 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7c90 (RPC#1305) tag(519002c) sent to node 1 [ CSSD]2013-05-23 16:15:30.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:30.044 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1305 (0x7f431bfe7c90) [ CSSD]2013-05-23 16:15:30.044 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7c90 (RPC#1305) state 6, flags 0x202 [ CSSD]2013-05-23 16:15:30.044 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7c90, ret 0, client (nil) [ CSSD]2013-05-23 16:15:30.044 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1305 [ CSSD]2013-05-23 16:15:30.415 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:30.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:30.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:32.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:32.472 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:32.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:33.259 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:33.259 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:15:34.512 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:34.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:34.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:34.528 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 16:15:34.529 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:15:34.529 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 16:15:34.529 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c048500) pid(6490) version 11:1:1:4 [ CSSD]2013-05-23 16:15:34.535 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c048500), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:15:34.536 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:15:34.536 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12ba00 with con 0x7f431c0e6b70, requested num -1 [ CSSD]2013-05-23 16:15:34.536 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:34.536 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1306 (0x7f431bfe7d10) [ CSSD]2013-05-23 16:15:34.536 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7d10 (RPC#1306) tag(51a002c) sent to node 1 [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(051a002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014451, gin 1, member 0 [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c147ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7d10 (RPC#1306) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7d10, ret 0, client 0x7f431c12ba00 member 0x7f431c13f950 [ CSSD]2013-05-23 16:15:34.536 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12ba00 [ CSSD]2013-05-23 16:15:34.537 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1306 [ CSSD]2013-05-23 16:15:34.543 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:15:34.543 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12ba00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:34.543 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:34.543 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1307 (0x7f431bfe7d90) [ CSSD]2013-05-23 16:15:34.543 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7d90 (RPC#1307) tag(51b002c) sent to node 1 [ CSSD]2013-05-23 16:15:34.544 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(051b002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:34.544 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:34.544 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7d90 (RPC#1307) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:34.544 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7d90, ret 0, client 0x7f431c12ba00 member 0x7f431c13f950 [ CSSD]2013-05-23 16:15:34.544 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1307 [ CSSD]2013-05-23 16:15:34.544 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:15:34.746 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 16:15:34.746 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c11ab50) skgpid ospid 6490 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:36.518 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:36.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:36.531 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:38.263 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:38.263 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:38.528 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:38.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:38.555 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:40.071 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:15:40.071 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1308 (0x7f431bfe7e10) [ CSSD]2013-05-23 16:15:40.071 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7e10 (RPC#1308) tag(51c002c) sent to node 1 [ CSSD]2013-05-23 16:15:40.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:40.072 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1308 (0x7f431bfe7e10) [ CSSD]2013-05-23 16:15:40.072 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7e10 (RPC#1308) state 6, flags 0x202 [ CSSD]2013-05-23 16:15:40.072 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7e10, ret 0, client (nil) [ CSSD]2013-05-23 16:15:40.072 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1308 [ CSSD]2013-05-23 16:15:40.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:40.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:40.649 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:42.586 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:42.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:42.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:43.266 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:43.267 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:44.633 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:44.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:44.746 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:45.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:45.048 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014452, gin 1, member 0 [ CSSD]2013-05-23 16:15:45.048 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:15:45.048 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:15:45.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:45.054 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:15:45.055 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:15:46.638 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:46.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:46.752 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:48.270 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:48.270 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:48.690 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:48.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:48.758 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:49.654 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 16:15:49.655 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:15:49.656 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 16:15:49.656 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6b70) proc(0x7f431c12f520) pid(6345/6345) version 11:1:1:4 [ CSSD]2013-05-23 16:15:49.671 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12f520), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:15:49.671 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:15:49.672 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:15:49.672 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:15:49.779 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147cb0) allocated [ CSSD]2013-05-23 16:15:49.780 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:15:49.780 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147cb0 - 1,2,3 [ CSSD]2013-05-23 16:15:49.780 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c147cb0) pid(6674) version 11:1:1:4 [ CSSD]2013-05-23 16:15:49.786 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c147cb0), client(1/0x7f431c150ab0) [ CSSD]2013-05-23 16:15:49.786 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c150ab0) [ CSSD]2013-05-23 16:15:49.786 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c150ab0 with con 0x7f431c13ba20, requested num -1 [ CSSD]2013-05-23 16:15:49.786 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:49.786 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1309 (0x7f431bfe7e90) [ CSSD]2013-05-23 16:15:49.786 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7e90 (RPC#1309) tag(51d002c) sent to node 1 [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(051d002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014453, gin 1, member 0 [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094ab0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7e90 (RPC#1309) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe7e90, ret 0, client 0x7f431c150ab0 member 0x7f431c147ba0 [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c150ab0 [ CSSD]2013-05-23 16:15:49.787 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1309 [ CSSD]2013-05-23 16:15:49.793 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c150ab0) [ CSSD]2013-05-23 16:15:49.793 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c150ab0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:49.793 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c147ba0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:15:49.793 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1310 (0x7f431bfe7f10) [ CSSD]2013-05-23 16:15:49.793 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7f10 (RPC#1310) tag(51e002c) sent to node 1 [ CSSD]2013-05-23 16:15:49.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(051e002c) incarnation(174631051) [ CSSD]2013-05-23 16:15:49.793 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:15:49.794 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7f10 (RPC#1310) state 6, flags 0x100 [ CSSD]2013-05-23 16:15:49.794 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe7f10, ret 0, client 0x7f431c150ab0 member 0x7f431c147ba0 [ CSSD]2013-05-23 16:15:49.794 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1310 [ CSSD]2013-05-23 16:15:49.794 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c147ba0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:15:49.996 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147cb0 [ CSSD]2013-05-23 16:15:49.996 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147cb0) con(0x7f431c0e6e20) skgpid ospid 6674 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:15:50.095 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:15:50.095 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1311 (0x7f431bfe7f90) [ CSSD]2013-05-23 16:15:50.096 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe7f90 (RPC#1311) tag(51f002c) sent to node 1 [ CSSD]2013-05-23 16:15:50.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:50.096 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1311 (0x7f431bfe7f90) [ CSSD]2013-05-23 16:15:50.096 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe7f90 (RPC#1311) state 6, flags 0x202 [ CSSD]2013-05-23 16:15:50.096 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe7f90, ret 0, client (nil) [ CSSD]2013-05-23 16:15:50.096 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1311 [ CSSD]2013-05-23 16:15:50.696 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:50.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:50.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:51.137 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147cb0) allocated [ CSSD]2013-05-23 16:15:51.138 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:15:51.138 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147cb0 - 1,2,3 [ CSSD]2013-05-23 16:15:51.138 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c147cb0) pid(6343/6343) version 11:1:1:4 [ CSSD]2013-05-23 16:15:51.154 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c147cb0), client(1/0x7f431c12ba00) [ CSSD]2013-05-23 16:15:51.154 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:15:51.154 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:15:51.154 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:15:52.707 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:52.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:52.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:53.273 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:53.273 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:54.713 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:54.827 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:54.827 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:56.720 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:56.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:56.851 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:15:58.277 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:15:58.277 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:15:58.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:15:58.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:15:58.864 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:00.119 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:16:00.119 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1312 (0x7f431bfe8010) [ CSSD]2013-05-23 16:16:00.120 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8010 (RPC#1312) tag(520002c) sent to node 1 [ CSSD]2013-05-23 16:16:00.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:00.120 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1312 (0x7f431bfe8010) [ CSSD]2013-05-23 16:16:00.120 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8010 (RPC#1312) state 6, flags 0x202 [ CSSD]2013-05-23 16:16:00.120 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8010, ret 0, client (nil) [ CSSD]2013-05-23 16:16:00.120 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1312 [ CSSD]2013-05-23 16:16:00.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:00.297 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014454, gin 1, member 0 [ CSSD]2013-05-23 16:16:00.297 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:16:00.297 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:16:00.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:00.303 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:16:00.303 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:16:00.755 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:00.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:00.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:02.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:02.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:02.875 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:03.281 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:03.281 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:04.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:04.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:04.928 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:05.040 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b360) allocated [ CSSD]2013-05-23 16:16:05.041 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:16:05.041 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b360 - 1,2,3 [ CSSD]2013-05-23 16:16:05.041 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13be70) proc(0x7f431c13b360) pid(6843) version 11:1:1:4 [ CSSD]2013-05-23 16:16:05.047 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13b360), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:16:05.047 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:05.047 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c13b5c0, requested num -1 [ CSSD]2013-05-23 16:16:05.047 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:05.047 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1313 (0x7f431bfe8090) [ CSSD]2013-05-23 16:16:05.048 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8090 (RPC#1313) tag(521002c) sent to node 1 [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0521002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014455, gin 1, member 0 [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133c10/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8090 (RPC#1313) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8090, ret 0, client 0x7f431c109f80 member 0x7f431c13b240 [ CSSD]2013-05-23 16:16:05.048 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:16:05.049 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1313 [ CSSD]2013-05-23 16:16:05.055 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:05.055 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:05.055 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b240) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:05.055 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1314 (0x7f431bfe8110) [ CSSD]2013-05-23 16:16:05.055 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8110 (RPC#1314) tag(522002c) sent to node 1 [ CSSD]2013-05-23 16:16:05.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0522002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:05.055 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:05.055 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8110 (RPC#1314) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:05.056 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8110, ret 0, client 0x7f431c109f80 member 0x7f431c13b240 [ CSSD]2013-05-23 16:16:05.056 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1314 [ CSSD]2013-05-23 16:16:05.056 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b240) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:16:05.258 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b360 [ CSSD]2013-05-23 16:16:05.258 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b360) con(0x7f431c13be70) skgpid ospid 6843 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:06.322 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12f520), client(2/0x7f431c150ab0) [ CSSD]2013-05-23 16:16:06.322 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c150ab0) [ CSSD]2013-05-23 16:16:06.322 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:16:06.322 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:16:06.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:06.972 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:06.972 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:07.283 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:07.284 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:16:08.809 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:09.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:09.025 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:10.149 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:16:10.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1315 (0x7f431bfe8190) [ CSSD]2013-05-23 16:16:10.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8190 (RPC#1315) tag(523002c) sent to node 1 [ CSSD]2013-05-23 16:16:10.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:10.150 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1315 (0x7f431bfe8190) [ CSSD]2013-05-23 16:16:10.150 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8190 (RPC#1315) state 6, flags 0x202 [ CSSD]2013-05-23 16:16:10.150 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8190, ret 0, client (nil) [ CSSD]2013-05-23 16:16:10.150 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1315 [ CSSD]2013-05-23 16:16:10.856 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:11.031 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:11.031 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:12.287 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:12.287 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:12.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:13.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:13.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:14.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:14.246 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014456, gin 1, member 0 [ CSSD]2013-05-23 16:16:14.246 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:16:14.246 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:16:14.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:14.253 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:16:14.253 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:16:14.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:15.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:15.047 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:15.546 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:15.546 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014457, gin 1, member 0 [ CSSD]2013-05-23 16:16:15.546 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:16:15.546 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:16:15.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:15.552 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:16:15.552 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:16:16.902 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:16.990 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b650) allocated [ CSSD]2013-05-23 16:16:16.992 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:16:16.992 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b650 - 1,2,3 [ CSSD]2013-05-23 16:16:16.992 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095440) proc(0x7f431c13b650) pid(6965) version 11:1:1:4 [ CSSD]2013-05-23 16:16:16.998 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13b650), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:16:16.998 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:16.998 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c109f80 with con 0x7f431c133b40, requested num -1 [ CSSD]2013-05-23 16:16:16.998 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:16:16.998 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1316 (0x7f431bfe8210) [ CSSD]2013-05-23 16:16:16.998 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8210 (RPC#1316) tag(524002c) sent to node 1 [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0524002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014458, gin 1, member 0 [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134120/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8210 (RPC#1316) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8210, ret 0, client 0x7f431c109f80 member 0x7f431c1338a0 [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:16:16.999 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1316 [ CSSD]2013-05-23 16:16:17.005 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:17.005 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:16:17.005 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1338a0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:16:17.005 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1317 (0x7f431bfe8290) [ CSSD]2013-05-23 16:16:17.005 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8290 (RPC#1317) tag(525002c) sent to node 1 [ CSSD]2013-05-23 16:16:17.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0525002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:17.005 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:16:17.005 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8290 (RPC#1317) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:17.005 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8290, ret 0, client 0x7f431c109f80 member 0x7f431c1338a0 [ CSSD]2013-05-23 16:16:17.006 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1317 [ CSSD]2013-05-23 16:16:17.006 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c1338a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:16:17.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:17.072 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:17.103 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083a30) allocated [ CSSD]2013-05-23 16:16:17.104 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:16:17.104 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083a30 - 1,2,3 [ CSSD]2013-05-23 16:16:17.104 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c094d80) proc(0x7f431c083a30) pid(6972) version 11:1:1:4 [ CSSD]2013-05-23 16:16:17.117 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:16:17.118 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:16:17.120 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083a30 [ CSSD]2013-05-23 16:16:17.120 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083a30) con(0x7f431c094d80) skgpid ospid 6972 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:17.208 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b650 [ CSSD]2013-05-23 16:16:17.208 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b650) con(0x7f431c095440) skgpid ospid 6965 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:17.290 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:17.291 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:18.046 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1337b0) allocated [ CSSD]2013-05-23 16:16:18.048 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:16:18.048 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1337b0 - 1,2,3 [ CSSD]2013-05-23 16:16:18.048 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095310) proc(0x7f431c1337b0) pid(6986) version 11:1:1:4 [ CSSD]2013-05-23 16:16:18.054 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1337b0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:16:18.054 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:18.054 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c13b700, requested num -1 [ CSSD]2013-05-23 16:16:18.054 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:18.054 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1318 (0x7f431bfe8310) [ CSSD]2013-05-23 16:16:18.054 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8310 (RPC#1318) tag(526002c) sent to node 1 [ CSSD]2013-05-23 16:16:18.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0526002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:18.055 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014459, gin 1, member 0 [ CSSD]2013-05-23 16:16:18.055 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133fd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:16:18.056 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:16:18.056 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:16:18.056 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8310 (RPC#1318) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:18.056 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8310, ret 0, client 0x7f431c109f80 member 0x7f431c1342d0 [ CSSD]2013-05-23 16:16:18.056 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:16:18.056 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1318 [ CSSD]2013-05-23 16:16:18.062 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:18.062 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:18.062 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1342d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:18.062 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1319 (0x7f431bfe8390) [ CSSD]2013-05-23 16:16:18.062 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8390 (RPC#1319) tag(527002c) sent to node 1 [ CSSD]2013-05-23 16:16:18.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0527002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:18.063 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:18.063 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8390 (RPC#1319) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:18.063 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8390, ret 0, client 0x7f431c109f80 member 0x7f431c1342d0 [ CSSD]2013-05-23 16:16:18.063 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1319 [ CSSD]2013-05-23 16:16:18.063 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1342d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:16:18.265 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1337b0 [ CSSD]2013-05-23 16:16:18.265 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1337b0) con(0x7f431c095310) skgpid ospid 6986 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:18.910 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:19.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:19.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:20.174 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:16:20.174 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1320 (0x7f431bfe8410) [ CSSD]2013-05-23 16:16:20.175 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8410 (RPC#1320) tag(528002c) sent to node 1 [ CSSD]2013-05-23 16:16:20.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:20.175 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1320 (0x7f431bfe8410) [ CSSD]2013-05-23 16:16:20.175 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8410 (RPC#1320) state 6, flags 0x202 [ CSSD]2013-05-23 16:16:20.175 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8410, ret 0, client (nil) [ CSSD]2013-05-23 16:16:20.175 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1320 [ CSSD]2013-05-23 16:16:20.288 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1337b0) allocated [ CSSD]2013-05-23 16:16:20.289 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:16:20.289 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1337b0 - 1,2,3 [ CSSD]2013-05-23 16:16:20.289 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095310) proc(0x7f431c1337b0) pid(7010) version 11:1:1:4 [ CSSD]2013-05-23 16:16:20.294 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1337b0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:16:20.295 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:20.295 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c13b700, requested num -1 [ CSSD]2013-05-23 16:16:20.295 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:20.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1321 (0x7f431bfe8490) [ CSSD]2013-05-23 16:16:20.295 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8490 (RPC#1321) tag(529002c) sent to node 1 [ CSSD]2013-05-23 16:16:20.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0529002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:20.295 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014460, gin 1, member 0 [ CSSD]2013-05-23 16:16:20.295 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133fd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:16:20.295 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:16:20.295 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:16:20.295 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8490 (RPC#1321) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:20.296 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8490, ret 0, client 0x7f431c109f80 member 0x7f431c1342d0 [ CSSD]2013-05-23 16:16:20.296 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:16:20.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1321 [ CSSD]2013-05-23 16:16:20.301 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:20.301 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:20.301 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1342d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:20.301 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1322 (0x7f431bfe8510) [ CSSD]2013-05-23 16:16:20.302 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8510 (RPC#1322) tag(52a002c) sent to node 1 [ CSSD]2013-05-23 16:16:20.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(052a002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:20.302 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:20.302 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8510 (RPC#1322) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:20.302 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8510, ret 0, client 0x7f431c109f80 member 0x7f431c1342d0 [ CSSD]2013-05-23 16:16:20.302 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1322 [ CSSD]2013-05-23 16:16:20.302 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1342d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:16:20.504 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1337b0 [ CSSD]2013-05-23 16:16:20.504 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1337b0) con(0x7f431c095310) skgpid ospid 7010 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:20.935 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:21.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:21.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:22.294 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:22.294 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:23.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:23.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:23.087 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:23.364 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0e5910) [ CSSD]2013-05-23 16:16:23.365 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0e5910), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:16:23.365 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0e5910), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:16:23.365 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:16:23.374 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:16:23.374 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:16:23.374 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:16:23.374 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:16:23.374 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:16:23.374 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c12f0c0) skgpid 1362 ospid 1362 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:25.014 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:25.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:25.139 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:26.297 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:26.297 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:16:27.021 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:27.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:27.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:29.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:29.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:29.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:30.198 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:16:30.199 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1323 (0x7f431bfe8590) [ CSSD]2013-05-23 16:16:30.199 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8590 (RPC#1323) tag(52b002c) sent to node 1 [ CSSD]2013-05-23 16:16:30.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:30.199 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1323 (0x7f431bfe8590) [ CSSD]2013-05-23 16:16:30.199 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8590 (RPC#1323) state 6, flags 0x202 [ CSSD]2013-05-23 16:16:30.199 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8590, ret 0, client (nil) [ CSSD]2013-05-23 16:16:30.199 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1323 [ CSSD]2013-05-23 16:16:30.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:30.807 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014461, gin 1, member 0 [ CSSD]2013-05-23 16:16:30.807 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:16:30.807 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:16:30.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:30.813 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:16:30.813 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:16:31.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:31.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:31.162 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:31.301 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:31.301 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:33.109 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:33.187 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:33.187 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:35.135 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:35.238 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:35.238 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:35.540 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135e90) allocated [ CSSD]2013-05-23 16:16:35.541 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:16:35.542 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135e90 - 1,2,3 [ CSSD]2013-05-23 16:16:35.542 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c135e90) pid(7180) version 11:1:1:4 [ CSSD]2013-05-23 16:16:35.547 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c135e90), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:16:35.547 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:16:35.547 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:16:35.547 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:35.547 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1324 (0x7f431bfe8610) [ CSSD]2013-05-23 16:16:35.548 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8610 (RPC#1324) tag(52c002c) sent to node 1 [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(052c002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014462, gin 1, member 0 [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c094d80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8610 (RPC#1324) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8610, ret 0, client 0x7f431c0e5910 member 0x7f431c0958d0 [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:16:35.548 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1324 [ CSSD]2013-05-23 16:16:35.555 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:16:35.555 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:35.555 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0958d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:35.555 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1325 (0x7f431bfe8690) [ CSSD]2013-05-23 16:16:35.555 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8690 (RPC#1325) tag(52d002c) sent to node 1 [ CSSD]2013-05-23 16:16:35.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(052d002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:35.555 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:35.555 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8690 (RPC#1325) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:35.555 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8690, ret 0, client 0x7f431c0e5910 member 0x7f431c0958d0 [ CSSD]2013-05-23 16:16:35.555 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1325 [ CSSD]2013-05-23 16:16:35.555 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0958d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:16:35.673 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b290) allocated [ CSSD]2013-05-23 16:16:35.674 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:16:35.675 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b290 - 1,2,3 [ CSSD]2013-05-23 16:16:35.675 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c13b290) pid(7089/7089) version 11:1:1:4 [ CSSD]2013-05-23 16:16:35.690 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13b290), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:16:35.690 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:16:35.690 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:16:35.690 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:16:35.758 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135e90 [ CSSD]2013-05-23 16:16:35.758 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135e90) con(0x7f431c0840c0) skgpid ospid 7180 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:36.304 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:36.305 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:37.141 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:37.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:37.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:39.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:39.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:39.257 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:40.223 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:16:40.223 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1326 (0x7f431bfe8710) [ CSSD]2013-05-23 16:16:40.223 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8710 (RPC#1326) tag(52e002c) sent to node 1 [ CSSD]2013-05-23 16:16:40.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:40.224 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1326 (0x7f431bfe8710) [ CSSD]2013-05-23 16:16:40.224 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8710 (RPC#1326) state 6, flags 0x202 [ CSSD]2013-05-23 16:16:40.224 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8710, ret 0, client (nil) [ CSSD]2013-05-23 16:16:40.224 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1326 [ CSSD]2013-05-23 16:16:41.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:41.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:41.262 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:41.308 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:41.308 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:43.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:43.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:43.288 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:45.214 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:45.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:45.360 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:46.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:46.067 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014463, gin 1, member 0 [ CSSD]2013-05-23 16:16:46.067 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:16:46.067 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:16:46.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:46.073 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:16:46.073 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:16:46.312 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:46.312 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:46.999 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13b290), client(2/0x7f431c0e5910) [ CSSD]2013-05-23 16:16:46.999 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0e5910) [ CSSD]2013-05-23 16:16:46.999 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:16:46.999 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:16:47.242 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:47.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:47.365 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:49.242 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:49.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:49.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:50.247 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:16:50.248 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1327 (0x7f431bfe8790) [ CSSD]2013-05-23 16:16:50.248 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8790 (RPC#1327) tag(52f002c) sent to node 1 [ CSSD]2013-05-23 16:16:50.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:50.248 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1327 (0x7f431bfe8790) [ CSSD]2013-05-23 16:16:50.248 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8790 (RPC#1327) state 6, flags 0x202 [ CSSD]2013-05-23 16:16:50.248 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8790, ret 0, client (nil) [ CSSD]2013-05-23 16:16:50.249 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1327 [ CSSD]2013-05-23 16:16:50.790 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:16:50.791 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:16:50.792 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:16:50.792 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12b850) proc(0x7f431c133a90) pid(7332) version 11:1:1:4 [ CSSD]2013-05-23 16:16:50.797 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:16:50.797 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:16:50.797 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135f60, requested num -1 [ CSSD]2013-05-23 16:16:50.797 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:50.797 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1328 (0x7f431bfe8810) [ CSSD]2013-05-23 16:16:50.797 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8810 (RPC#1328) tag(530002c) sent to node 1 [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0530002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014464, gin 1, member 0 [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134290/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8810 (RPC#1328) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8810, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:16:50.798 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1328 [ CSSD]2013-05-23 16:16:50.805 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:16:50.805 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:50.805 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133970) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:16:50.805 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1329 (0x7f431bfe8890) [ CSSD]2013-05-23 16:16:50.805 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8890 (RPC#1329) tag(531002c) sent to node 1 [ CSSD]2013-05-23 16:16:50.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0531002c) incarnation(174631051) [ CSSD]2013-05-23 16:16:50.805 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:16:50.805 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8890 (RPC#1329) state 6, flags 0x100 [ CSSD]2013-05-23 16:16:50.805 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8890, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:16:50.805 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1329 [ CSSD]2013-05-23 16:16:50.805 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133970) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:16:51.008 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:16:51.008 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c12b850) skgpid ospid 7332 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:16:51.249 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:51.315 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:51.316 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:51.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:51.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:53.301 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:53.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:53.408 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:55.347 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:55.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:55.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:56.319 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:16:56.319 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:16:57.398 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:57.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:57.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:16:59.404 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:16:59.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:16:59.477 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:00.271 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:17:00.272 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1330 (0x7f431bfe8910) [ CSSD]2013-05-23 16:17:00.272 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8910 (RPC#1330) tag(532002c) sent to node 1 [ CSSD]2013-05-23 16:17:00.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:00.272 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1330 (0x7f431bfe8910) [ CSSD]2013-05-23 16:17:00.272 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8910 (RPC#1330) state 6, flags 0x202 [ CSSD]2013-05-23 16:17:00.272 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8910, ret 0, client (nil) [ CSSD]2013-05-23 16:17:00.273 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1330 [ CSSD]2013-05-23 16:17:00.322 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:00.323 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:17:01.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:01.328 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014465, gin 1, member 0 [ CSSD]2013-05-23 16:17:01.328 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:01.328 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:01.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:01.334 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:01.334 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:01.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:01.483 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:01.483 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:03.420 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:03.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:03.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:05.326 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:05.326 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:05.466 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:05.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:05.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:06.041 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:17:06.042 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:17:06.043 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:17:06.043 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e69d0) proc(0x7f431c133a90) pid(7497) version 11:1:1:4 [ CSSD]2013-05-23 16:17:06.049 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:17:06.049 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:06.049 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135f60, requested num -1 [ CSSD]2013-05-23 16:17:06.049 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:06.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1331 (0x7f431bfe8990) [ CSSD]2013-05-23 16:17:06.049 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8990 (RPC#1331) tag(533002c) sent to node 1 [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0533002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014466, gin 1, member 0 [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134250/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8990 (RPC#1331) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8990, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:17:06.050 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1331 [ CSSD]2013-05-23 16:17:06.055 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:06.055 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:06.055 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133970) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:06.055 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1332 (0x7f431bfe8a10) [ CSSD]2013-05-23 16:17:06.055 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8a10 (RPC#1332) tag(534002c) sent to node 1 [ CSSD]2013-05-23 16:17:06.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0534002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:06.056 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:06.056 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8a10 (RPC#1332) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:06.056 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8a10, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:06.056 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1332 [ CSSD]2013-05-23 16:17:06.056 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133970) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:17:06.259 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:17:06.259 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c0e69d0) skgpid ospid 7497 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:07.499 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:07.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:07.586 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:09.505 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:09.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:09.638 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:10.298 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:17:10.299 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1333 (0x7f431bfe8a90) [ CSSD]2013-05-23 16:17:10.299 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8a90 (RPC#1333) tag(535002c) sent to node 1 [ CSSD]2013-05-23 16:17:10.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:10.299 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1333 (0x7f431bfe8a90) [ CSSD]2013-05-23 16:17:10.299 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8a90 (RPC#1333) state 6, flags 0x202 [ CSSD]2013-05-23 16:17:10.299 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8a90, ret 0, client (nil) [ CSSD]2013-05-23 16:17:10.300 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1333 [ CSSD]2013-05-23 16:17:10.329 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:10.330 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:11.531 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:11.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:11.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:11.888 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:11.888 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014467, gin 1, member 0 [ CSSD]2013-05-23 16:17:11.888 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:11.888 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:11.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:11.893 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:11.893 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:13.541 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:13.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:13.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:14.505 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:14.505 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014468, gin 1, member 0 [ CSSD]2013-05-23 16:17:14.505 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:14.505 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:14.512 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:14.512 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:14.512 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:15.333 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:15.333 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:15.452 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:17:15.453 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:17:15.454 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:17:15.454 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e69d0) proc(0x7f431c133a90) pid(7802) version 11:1:1:4 [ CSSD]2013-05-23 16:17:15.547 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:15.581 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1361c0) allocated [ CSSD]2013-05-23 16:17:15.582 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:17:15.583 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1361c0 - 1,2,3 [ CSSD]2013-05-23 16:17:15.583 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134200) proc(0x7f431c1361c0) pid(7813) version 11:1:1:4 [ CSSD]2013-05-23 16:17:15.599 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:17:15.601 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:17:15.603 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1361c0 [ CSSD]2013-05-23 16:17:15.603 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1361c0) con(0x7f431c134200) skgpid ospid 7813 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:15.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:15.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:15.674 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(66/0x7f431c0cf720) [ CSSD]2013-05-23 16:17:15.674 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 66 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:15.674 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c0cf720 with con 0x7f431c135f60, requested num -1 [ CSSD]2013-05-23 16:17:15.674 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:17:15.674 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1334 (0x7f431bfe8b10) [ CSSD]2013-05-23 16:17:15.675 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8b10 (RPC#1334) tag(536002c) sent to node 1 [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0536002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014469, gin 1, member 0 [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c134030/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8b10 (RPC#1334) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8b10, ret 0, client 0x7f431c0cf720 member 0x7f431c134420 [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:17:15.675 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1334 [ CSSD]2013-05-23 16:17:15.681 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 66 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:15.681 [1191196992] >TRACE: clssgmExitGrock: client 66 (0x7f431c0cf720), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:17:15.681 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134420) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:17:15.681 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1335 (0x7f431bfe8b90) [ CSSD]2013-05-23 16:17:15.681 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8b90 (RPC#1335) tag(537002c) sent to node 1 [ CSSD]2013-05-23 16:17:15.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0537002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:15.682 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:17:15.682 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8b90 (RPC#1335) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:15.682 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8b90, ret 0, client 0x7f431c0cf720 member 0x7f431c134420 [ CSSD]2013-05-23 16:17:15.682 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1335 [ CSSD]2013-05-23 16:17:15.682 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c134420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:17:15.683 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:17:15.683 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c0e69d0) skgpid ospid 7802 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:16.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:16.575 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014470, gin 1, member 0 [ CSSD]2013-05-23 16:17:16.575 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:16.575 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:16.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:16.582 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:16.582 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:17.599 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:17.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:17.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:18.309 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:17:18.311 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:17:18.311 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:17:18.311 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e69d0) proc(0x7f431c133a90) pid(7841) version 11:1:1:4 [ CSSD]2013-05-23 16:17:18.318 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:17:18.318 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:18.318 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135f60, requested num -1 [ CSSD]2013-05-23 16:17:18.318 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:18.318 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1336 (0x7f431bfe8c10) [ CSSD]2013-05-23 16:17:18.318 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8c10 (RPC#1336) tag(538002c) sent to node 1 [ CSSD]2013-05-23 16:17:18.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0538002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:18.318 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014471, gin 1, member 0 [ CSSD]2013-05-23 16:17:18.318 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134250/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:17:18.319 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:17:18.319 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:17:18.319 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8c10 (RPC#1336) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:18.319 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8c10, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:18.319 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:17:18.319 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1336 [ CSSD]2013-05-23 16:17:18.326 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:18.326 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:18.326 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133970) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:18.326 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1337 (0x7f431bfe8c90) [ CSSD]2013-05-23 16:17:18.326 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8c90 (RPC#1337) tag(539002c) sent to node 1 [ CSSD]2013-05-23 16:17:18.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0539002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:18.326 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:18.326 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8c90 (RPC#1337) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:18.326 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8c90, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:18.327 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1337 [ CSSD]2013-05-23 16:17:18.327 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133970) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:17:18.529 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:17:18.529 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c0e69d0) skgpid ospid 7841 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:19.605 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:19.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:19.679 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:20.325 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:17:20.326 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1338 (0x7f431bfe8d10) [ CSSD]2013-05-23 16:17:20.326 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8d10 (RPC#1338) tag(53a002c) sent to node 1 [ CSSD]2013-05-23 16:17:20.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:20.326 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1338 (0x7f431bfe8d10) [ CSSD]2013-05-23 16:17:20.326 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8d10 (RPC#1338) state 6, flags 0x202 [ CSSD]2013-05-23 16:17:20.326 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8d10, ret 0, client (nil) [ CSSD]2013-05-23 16:17:20.326 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1338 [ CSSD]2013-05-23 16:17:20.336 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:20.336 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:21.308 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:17:21.309 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:17:21.309 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:17:21.309 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12b850) proc(0x7f431c133a90) pid(7870) version 11:1:1:4 [ CSSD]2013-05-23 16:17:21.315 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:17:21.315 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:21.315 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135f60, requested num -1 [ CSSD]2013-05-23 16:17:21.315 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:21.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1339 (0x7f431bfe8d90) [ CSSD]2013-05-23 16:17:21.315 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8d90 (RPC#1339) tag(53b002c) sent to node 1 [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(053b002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014472, gin 1, member 0 [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1341e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8d90 (RPC#1339) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8d90, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:17:21.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1339 [ CSSD]2013-05-23 16:17:21.321 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:21.321 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:21.321 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133970) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:21.322 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1340 (0x7f431bfe8e10) [ CSSD]2013-05-23 16:17:21.322 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8e10 (RPC#1340) tag(53c002c) sent to node 1 [ CSSD]2013-05-23 16:17:21.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(053c002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:21.322 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:21.322 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8e10 (RPC#1340) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:21.322 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8e10, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:21.322 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1340 [ CSSD]2013-05-23 16:17:21.322 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133970) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:17:21.525 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:17:21.525 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c12b850) skgpid ospid 7870 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:21.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:21.684 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:21.684 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:23.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:23.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:23.731 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:25.339 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:25.340 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:25.647 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:25.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:25.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:27.700 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:27.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:27.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:29.707 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:29.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:29.759 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:30.343 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:30.343 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:30.350 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:17:30.351 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1341 (0x7f431bfe8e90) [ CSSD]2013-05-23 16:17:30.351 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8e90 (RPC#1341) tag(53d002c) sent to node 1 [ CSSD]2013-05-23 16:17:30.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:30.351 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1341 (0x7f431bfe8e90) [ CSSD]2013-05-23 16:17:30.351 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8e90 (RPC#1341) state 6, flags 0x202 [ CSSD]2013-05-23 16:17:30.351 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe8e90, ret 0, client (nil) [ CSSD]2013-05-23 16:17:30.351 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1341 [ CSSD]2013-05-23 16:17:31.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:31.765 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:31.765 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:31.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:31.824 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014473, gin 1, member 0 [ CSSD]2013-05-23 16:17:31.824 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:31.824 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:31.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:31.831 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:31.831 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:33.722 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:33.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:33.770 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:34.346 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:34.347 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:17:35.728 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:35.780 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:35.780 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:36.554 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:17:36.555 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:17:36.555 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:17:36.555 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12b850) proc(0x7f431c133a90) pid(8038) version 11:1:1:4 [ CSSD]2013-05-23 16:17:36.560 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:17:36.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:36.560 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135f60, requested num -1 [ CSSD]2013-05-23 16:17:36.560 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:36.560 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1342 (0x7f431bfe8f10) [ CSSD]2013-05-23 16:17:36.560 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8f10 (RPC#1342) tag(53e002c) sent to node 1 [ CSSD]2013-05-23 16:17:36.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(053e002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:36.560 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014474, gin 1, member 0 [ CSSD]2013-05-23 16:17:36.560 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1341e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:17:36.561 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:17:36.561 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:17:36.561 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8f10 (RPC#1342) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:36.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe8f10, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:36.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:17:36.561 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1342 [ CSSD]2013-05-23 16:17:36.565 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:36.565 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:36.565 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133970) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:36.565 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1343 (0x7f431bfe8f90) [ CSSD]2013-05-23 16:17:36.565 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe8f90 (RPC#1343) tag(53f002c) sent to node 1 [ CSSD]2013-05-23 16:17:36.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(053f002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:36.565 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:36.565 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe8f90 (RPC#1343) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:36.565 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe8f90, ret 0, client 0x7f431c0cf720 member 0x7f431c133970 [ CSSD]2013-05-23 16:17:36.566 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1343 [ CSSD]2013-05-23 16:17:36.566 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133970) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:17:36.768 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:17:36.768 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c12b850) skgpid ospid 8038 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:37.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:37.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:37.805 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:39.350 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:39.350 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:39.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:39.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:39.825 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:40.389 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:17:40.390 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1344 (0x7f431bfe9010) [ CSSD]2013-05-23 16:17:40.390 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9010 (RPC#1344) tag(540002c) sent to node 1 [ CSSD]2013-05-23 16:17:40.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:40.391 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1344 (0x7f431bfe9010) [ CSSD]2013-05-23 16:17:40.391 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9010 (RPC#1344) state 6, flags 0x202 [ CSSD]2013-05-23 16:17:40.391 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9010, ret 0, client (nil) [ CSSD]2013-05-23 16:17:40.391 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1344 [ CSSD]2013-05-23 16:17:40.925 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133a90) allocated [ CSSD]2013-05-23 16:17:40.926 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:17:40.927 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133a90 - 1,2,3 [ CSSD]2013-05-23 16:17:40.927 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12b850) proc(0x7f431c133a90) pid(6335/6335) version 11:1:1:4 [ CSSD]2013-05-23 16:17:40.943 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:17:40.943 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:17:40.943 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:17:40.943 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:17:41.012 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c133a90), client(2/0x7f431c134470) [ CSSD]2013-05-23 16:17:41.012 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c134470) [ CSSD]2013-05-23 16:17:41.012 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:17:41.012 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 28 [ CSSD]2013-05-23 16:17:41.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:41.396 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014475, gin 1, member 0 [ CSSD]2013-05-23 16:17:41.396 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:41.396 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:41.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:41.401 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:41.401 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:41.537 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c147cb0), client(2/0x7f431c134760) [ CSSD]2013-05-23 16:17:41.537 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c134760) [ CSSD]2013-05-23 16:17:41.537 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:17:41.537 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 16:17:41.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:41.825 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:41.831 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:43.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:43.847 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:43.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:44.353 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:44.353 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:45.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:45.853 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:45.867 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:47.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:47.085 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014476, gin 1, member 0 [ CSSD]2013-05-23 16:17:47.085 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:17:47.085 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:17:47.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:47.124 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:17:47.124 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:17:47.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:47.859 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:47.900 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:49.094 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:17:49.094 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:17:49.094 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:17:49.094 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:17:49.094 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147f40 [ CSSD]2013-05-23 16:17:49.094 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147f40) con(0x7f431c0e6720) skgpid 2545 ospid 2545 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:49.356 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:49.356 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:49.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:49.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:49.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:50.414 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:17:50.414 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1345 (0x7f431bfe9090) [ CSSD]2013-05-23 16:17:50.414 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9090 (RPC#1345) tag(541002c) sent to node 1 [ CSSD]2013-05-23 16:17:50.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:50.415 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1345 (0x7f431bfe9090) [ CSSD]2013-05-23 16:17:50.415 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9090 (RPC#1345) state 6, flags 0x202 [ CSSD]2013-05-23 16:17:50.415 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9090, ret 0, client (nil) [ CSSD]2013-05-23 16:17:50.415 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1345 [ CSSD]2013-05-23 16:17:51.800 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:17:51.801 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:17:51.801 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:17:51.801 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c134c40) pid(8239) version 11:1:1:4 [ CSSD]2013-05-23 16:17:51.807 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:17:51.807 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:17:51.807 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:17:51.807 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:51.807 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1346 (0x7f431bfe9110) [ CSSD]2013-05-23 16:17:51.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9110 (RPC#1346) tag(542002c) sent to node 1 [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0542002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014477, gin 1, member 0 [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082530/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9110 (RPC#1346) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9110, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:17:51.808 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1346 [ CSSD]2013-05-23 16:17:51.814 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:17:51.814 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:51.814 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:17:51.814 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1347 (0x7f431bfe9190) [ CSSD]2013-05-23 16:17:51.814 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9190 (RPC#1347) tag(543002c) sent to node 1 [ CSSD]2013-05-23 16:17:51.814 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0543002c) incarnation(174631051) [ CSSD]2013-05-23 16:17:51.814 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:17:51.814 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9190 (RPC#1347) state 6, flags 0x100 [ CSSD]2013-05-23 16:17:51.814 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9190, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:17:51.815 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1347 [ CSSD]2013-05-23 16:17:51.815 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:17:51.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:51.875 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:51.912 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:52.017 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:17:52.017 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c0e6720) skgpid ospid 8239 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:53.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:53.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:53.948 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:54.359 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:54.359 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:17:54.554 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:17:54.555 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:17:54.555 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:17:54.555 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134c40) pid(8315/8315) version 11:1:1:4 [ CSSD]2013-05-23 16:17:54.560 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:17:54.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:17:54.560 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:17:54.560 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:17:54.945 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:17:54.945 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:17:54.945 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:17:54.945 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:17:54.945 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:17:54.945 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c108e60) skgpid 8315 ospid 8315 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:17:55.947 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:55.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:55.994 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:58.000 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:17:58.000 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:17:58.001 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:17:59.362 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:17:59.362 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:00.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:00.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:00.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:00.438 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:18:00.438 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1348 (0x7f431bfe9210) [ CSSD]2013-05-23 16:18:00.439 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9210 (RPC#1348) tag(544002c) sent to node 1 [ CSSD]2013-05-23 16:18:00.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:00.439 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1348 (0x7f431bfe9210) [ CSSD]2013-05-23 16:18:00.439 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9210 (RPC#1348) state 6, flags 0x202 [ CSSD]2013-05-23 16:18:00.439 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9210, ret 0, client (nil) [ CSSD]2013-05-23 16:18:00.439 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1348 [ CSSD]2013-05-23 16:18:02.013 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:02.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:02.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:02.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:02.334 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014478, gin 1, member 0 [ CSSD]2013-05-23 16:18:02.334 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:18:02.335 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:18:02.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:02.341 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:18:02.341 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:18:04.023 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:04.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:04.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:04.366 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:04.366 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:06.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:06.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:06.093 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:07.049 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:18:07.050 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:18:07.050 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:18:07.050 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c134c40) pid(8404) version 11:1:1:4 [ CSSD]2013-05-23 16:18:07.055 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:18:07.055 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:07.056 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:18:07.056 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:07.056 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1349 (0x7f431bfe9290) [ CSSD]2013-05-23 16:18:07.056 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9290 (RPC#1349) tag(545002c) sent to node 1 [ CSSD]2013-05-23 16:18:07.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0545002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:07.056 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014479, gin 1, member 0 [ CSSD]2013-05-23 16:18:07.056 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082530/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:18:07.056 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:18:07.056 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:18:07.057 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9290 (RPC#1349) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:07.057 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9290, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:18:07.057 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:18:07.057 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1349 [ CSSD]2013-05-23 16:18:07.062 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:07.062 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:07.062 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:07.062 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1350 (0x7f431bfe9310) [ CSSD]2013-05-23 16:18:07.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9310 (RPC#1350) tag(546002c) sent to node 1 [ CSSD]2013-05-23 16:18:07.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0546002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:07.063 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:07.063 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9310 (RPC#1350) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:07.063 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9310, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:18:07.063 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1350 [ CSSD]2013-05-23 16:18:07.063 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:18:07.266 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:18:07.266 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c0e6720) skgpid ospid 8404 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:18:08.052 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:08.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:08.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:09.369 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:09.369 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:10.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:10.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:10.152 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:10.461 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:18:10.462 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1351 (0x7f431bfe9390) [ CSSD]2013-05-23 16:18:10.462 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9390 (RPC#1351) tag(547002c) sent to node 1 [ CSSD]2013-05-23 16:18:10.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:10.462 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1351 (0x7f431bfe9390) [ CSSD]2013-05-23 16:18:10.462 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9390 (RPC#1351) state 6, flags 0x202 [ CSSD]2013-05-23 16:18:10.462 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9390, ret 0, client (nil) [ CSSD]2013-05-23 16:18:10.463 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1351 [ CSSD]2013-05-23 16:18:12.082 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:12.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:12.158 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:14.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:14.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:14.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:14.372 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:14.372 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:14.765 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:14.765 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014480, gin 1, member 0 [ CSSD]2013-05-23 16:18:14.765 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:18:14.765 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:18:14.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:14.772 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:18:14.772 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:18:16.120 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:16.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:16.173 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:17.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:17.597 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014481, gin 1, member 0 [ CSSD]2013-05-23 16:18:17.598 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:18:17.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:18:17.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:17.604 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:18:17.604 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:18:18.132 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:18.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:18.178 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:18.567 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091490) allocated [ CSSD]2013-05-23 16:18:18.569 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:18:18.569 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091490 - 1,2,3 [ CSSD]2013-05-23 16:18:18.569 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c091490) pid(8533) version 11:1:1:4 [ CSSD]2013-05-23 16:18:18.575 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c091490), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:18:18.575 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:18.575 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:18:18.575 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:18.575 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1352 (0x7f431bfe9410) [ CSSD]2013-05-23 16:18:18.575 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9410 (RPC#1352) tag(548002c) sent to node 1 [ CSSD]2013-05-23 16:18:18.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0548002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:18.575 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014482, gin 1, member 0 [ CSSD]2013-05-23 16:18:18.575 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082530/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:18:18.576 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:18:18.576 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:18:18.576 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9410 (RPC#1352) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:18.576 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9410, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:18:18.576 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:18:18.576 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1352 [ CSSD]2013-05-23 16:18:18.581 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:18.581 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:18.581 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:18.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1353 (0x7f431bfe9490) [ CSSD]2013-05-23 16:18:18.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9490 (RPC#1353) tag(549002c) sent to node 1 [ CSSD]2013-05-23 16:18:18.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0549002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:18.582 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:18.582 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9490 (RPC#1353) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:18.582 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9490, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:18:18.582 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1353 [ CSSD]2013-05-23 16:18:18.582 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:18:18.784 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091490 [ CSSD]2013-05-23 16:18:18.784 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091490) con(0x7f431c108e60) skgpid ospid 8533 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:18:19.375 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:19.375 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:20.138 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:20.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:20.190 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:20.485 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:18:20.486 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1354 (0x7f431bfe9510) [ CSSD]2013-05-23 16:18:20.486 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9510 (RPC#1354) tag(54a002c) sent to node 1 [ CSSD]2013-05-23 16:18:20.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:20.486 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1354 (0x7f431bfe9510) [ CSSD]2013-05-23 16:18:20.486 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9510 (RPC#1354) state 6, flags 0x202 [ CSSD]2013-05-23 16:18:20.486 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9510, ret 0, client (nil) [ CSSD]2013-05-23 16:18:20.486 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1354 [ CSSD]2013-05-23 16:18:22.164 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:22.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:22.194 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:22.299 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:18:22.300 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:18:22.300 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:18:22.300 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c134c40) pid(8579) version 11:1:1:4 [ CSSD]2013-05-23 16:18:22.306 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:18:22.306 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:22.306 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:18:22.306 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:22.306 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1355 (0x7f431bfe9590) [ CSSD]2013-05-23 16:18:22.307 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9590 (RPC#1355) tag(54b002c) sent to node 1 [ CSSD]2013-05-23 16:18:22.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(054b002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:22.307 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014483, gin 1, member 0 [ CSSD]2013-05-23 16:18:22.307 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082530/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:18:22.307 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:18:22.307 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:18:22.308 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9590 (RPC#1355) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:22.308 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9590, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:18:22.308 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:18:22.308 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1355 [ CSSD]2013-05-23 16:18:22.314 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:22.314 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:22.314 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:22.314 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1356 (0x7f431bfe9610) [ CSSD]2013-05-23 16:18:22.314 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9610 (RPC#1356) tag(54c002c) sent to node 1 [ CSSD]2013-05-23 16:18:22.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(054c002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:22.315 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:22.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9610 (RPC#1356) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:22.315 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9610, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:18:22.315 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1356 [ CSSD]2013-05-23 16:18:22.315 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:18:22.517 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:18:22.517 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c0e6720) skgpid ospid 8579 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:18:24.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:24.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:24.249 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:24.379 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:24.379 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:26.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:26.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:26.254 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:28.253 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:28.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:28.259 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:29.382 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:29.382 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:30.259 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:30.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:30.271 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:30.508 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:18:30.509 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1357 (0x7f431bfe9690) [ CSSD]2013-05-23 16:18:30.509 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9690 (RPC#1357) tag(54d002c) sent to node 1 [ CSSD]2013-05-23 16:18:30.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:30.509 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1357 (0x7f431bfe9690) [ CSSD]2013-05-23 16:18:30.509 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9690 (RPC#1357) state 6, flags 0x202 [ CSSD]2013-05-23 16:18:30.509 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9690, ret 0, client (nil) [ CSSD]2013-05-23 16:18:30.509 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1357 [ CSSD]2013-05-23 16:18:32.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:32.276 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:32.285 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:32.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:32.847 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014484, gin 1, member 0 [ CSSD]2013-05-23 16:18:32.847 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:18:32.847 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:18:32.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:32.887 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:18:32.887 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:18:34.289 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:34.289 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:34.334 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:34.385 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:34.385 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:36.294 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:36.294 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:36.340 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:37.554 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:18:37.555 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:18:37.556 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:18:37.556 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c134c40) pid(8789) version 11:1:1:4 [ CSSD]2013-05-23 16:18:37.561 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:18:37.561 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:37.561 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f570, requested num -1 [ CSSD]2013-05-23 16:18:37.562 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:37.562 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1358 (0x7f431bfe9710) [ CSSD]2013-05-23 16:18:37.562 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9710 (RPC#1358) tag(54e002c) sent to node 1 [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(054e002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014485, gin 1, member 0 [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9710 (RPC#1358) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9710, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:18:37.562 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1358 [ CSSD]2013-05-23 16:18:37.568 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:37.568 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:37.568 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f360) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:37.568 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1359 (0x7f431bfe9790) [ CSSD]2013-05-23 16:18:37.568 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9790 (RPC#1359) tag(54f002c) sent to node 1 [ CSSD]2013-05-23 16:18:37.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(054f002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:37.568 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:37.569 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9790 (RPC#1359) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:37.569 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9790, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:18:37.569 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1359 [ CSSD]2013-05-23 16:18:37.569 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f360) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:18:37.771 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:18:37.771 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c0e6720) skgpid ospid 8789 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:18:38.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:38.057 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014486, gin 1, member 0 [ CSSD]2013-05-23 16:18:38.057 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:18:38.057 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:18:38.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:38.063 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:18:38.063 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:18:38.190 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:18:38.190 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:18:38.191 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:18:38.192 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:18:38.194 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:18:38.194 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:18:38.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:38.299 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:38.353 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:39.388 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:39.389 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:40.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:40.352 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:40.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:40.532 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:18:40.533 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1360 (0x7f431bfe9810) [ CSSD]2013-05-23 16:18:40.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9810 (RPC#1360) tag(550002c) sent to node 1 [ CSSD]2013-05-23 16:18:40.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:40.533 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1360 (0x7f431bfe9810) [ CSSD]2013-05-23 16:18:40.533 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9810 (RPC#1360) state 6, flags 0x202 [ CSSD]2013-05-23 16:18:40.533 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9810, ret 0, client (nil) [ CSSD]2013-05-23 16:18:40.533 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1360 [ CSSD]2013-05-23 16:18:42.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:42.358 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:42.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:44.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:44.369 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:44.392 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:44.392 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:44.417 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:46.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:46.375 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:46.423 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:48.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:48.095 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014487, gin 1, member 0 [ CSSD]2013-05-23 16:18:48.095 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:18:48.095 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:18:48.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:48.101 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:18:48.101 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:18:48.380 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:48.380 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:48.434 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:49.396 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:49.396 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:50.441 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:50.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:50.453 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:50.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:18:50.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1361 (0x7f431bfe9890) [ CSSD]2013-05-23 16:18:50.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9890 (RPC#1361) tag(551002c) sent to node 1 [ CSSD]2013-05-23 16:18:50.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:50.557 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1361 (0x7f431bfe9890) [ CSSD]2013-05-23 16:18:50.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9890 (RPC#1361) state 6, flags 0x202 [ CSSD]2013-05-23 16:18:50.557 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9890, ret 0, client (nil) [ CSSD]2013-05-23 16:18:50.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1361 [ CSSD]2013-05-23 16:18:52.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:52.478 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:52.511 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:52.808 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:18:52.810 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:18:52.810 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:18:52.810 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134c40) pid(8963) version 11:1:1:4 [ CSSD]2013-05-23 16:18:52.816 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:18:52.816 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:52.816 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f570, requested num -1 [ CSSD]2013-05-23 16:18:52.816 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:52.816 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1362 (0x7f431bfe9910) [ CSSD]2013-05-23 16:18:52.816 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9910 (RPC#1362) tag(552002c) sent to node 1 [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0552002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014488, gin 1, member 0 [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9910 (RPC#1362) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9910, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:18:52.817 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1362 [ CSSD]2013-05-23 16:18:52.824 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:18:52.824 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:52.824 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f360) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:18:52.824 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1363 (0x7f431bfe9990) [ CSSD]2013-05-23 16:18:52.824 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9990 (RPC#1363) tag(553002c) sent to node 1 [ CSSD]2013-05-23 16:18:52.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0553002c) incarnation(174631051) [ CSSD]2013-05-23 16:18:52.824 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:18:52.824 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9990 (RPC#1363) state 6, flags 0x100 [ CSSD]2013-05-23 16:18:52.824 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9990, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:18:52.825 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1363 [ CSSD]2013-05-23 16:18:52.825 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f360) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:18:53.027 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:18:53.027 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c108e60) skgpid ospid 8963 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:18:53.398 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:53.399 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:18:54.537 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:54.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:54.551 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:56.543 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:56.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:56.576 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:18:58.402 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:18:58.402 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:18:58.553 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:18:58.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:18:58.582 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:00.559 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:00.582 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:19:00.582 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1364 (0x7f431bfe9a10) [ CSSD]2013-05-23 16:19:00.583 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9a10 (RPC#1364) tag(554002c) sent to node 1 [ CSSD]2013-05-23 16:19:00.583 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:00.583 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1364 (0x7f431bfe9a10) [ CSSD]2013-05-23 16:19:00.583 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9a10 (RPC#1364) state 6, flags 0x202 [ CSSD]2013-05-23 16:19:00.583 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9a10, ret 0, client (nil) [ CSSD]2013-05-23 16:19:00.583 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1364 [ CSSD]2013-05-23 16:19:00.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:00.594 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:02.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:02.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:02.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:03.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:03.354 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014489, gin 1, member 0 [ CSSD]2013-05-23 16:19:03.354 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:19:03.354 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:19:03.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:03.361 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:19:03.361 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:19:03.405 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:03.406 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:04.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:04.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:04.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:06.644 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:06.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:06.667 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:08.059 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:19:08.061 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:19:08.061 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:19:08.061 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134c40) pid(9156) version 11:1:1:4 [ CSSD]2013-05-23 16:19:08.067 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:19:08.067 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:08.067 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:19:08.067 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:08.067 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1365 (0x7f431bfe9a90) [ CSSD]2013-05-23 16:19:08.068 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9a90 (RPC#1365) tag(555002c) sent to node 1 [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0555002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014490, gin 1, member 0 [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082530/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9a90 (RPC#1365) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9a90, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:19:08.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1365 [ CSSD]2013-05-23 16:19:08.074 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:08.074 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:08.074 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:08.075 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1366 (0x7f431bfe9b10) [ CSSD]2013-05-23 16:19:08.075 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9b10 (RPC#1366) tag(556002c) sent to node 1 [ CSSD]2013-05-23 16:19:08.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0556002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:08.075 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:08.075 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9b10 (RPC#1366) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:08.075 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9b10, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:19:08.075 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1366 [ CSSD]2013-05-23 16:19:08.075 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:19:08.278 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:19:08.278 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c108e60) skgpid ospid 9156 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:19:08.409 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:08.409 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:08.655 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:08.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:08.693 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:10.632 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:19:10.633 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1367 (0x7f431bfe9b90) [ CSSD]2013-05-23 16:19:10.633 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9b90 (RPC#1367) tag(557002c) sent to node 1 [ CSSD]2013-05-23 16:19:10.633 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:10.633 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1367 (0x7f431bfe9b90) [ CSSD]2013-05-23 16:19:10.633 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9b90 (RPC#1367) state 6, flags 0x202 [ CSSD]2013-05-23 16:19:10.633 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9b90, ret 0, client (nil) [ CSSD]2013-05-23 16:19:10.633 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1367 [ CSSD]2013-05-23 16:19:10.662 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:10.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:10.703 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:12.412 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:12.412 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:19:12.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:12.707 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:12.712 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:14.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:14.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:14.762 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:15.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:15.023 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014491, gin 1, member 0 [ CSSD]2013-05-23 16:19:15.023 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:19:15.023 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:19:15.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:15.030 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:19:15.030 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:19:16.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:16.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:16.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:17.415 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:17.416 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:18.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:18.602 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014492, gin 1, member 0 [ CSSD]2013-05-23 16:19:18.603 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:19:18.603 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:19:18.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:18.609 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:19:18.609 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:19:18.736 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:18.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:18.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:18.837 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091490) allocated [ CSSD]2013-05-23 16:19:18.838 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:19:18.839 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091490 - 1,2,3 [ CSSD]2013-05-23 16:19:18.839 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c091490) pid(9273) version 11:1:1:4 [ CSSD]2013-05-23 16:19:18.845 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c091490), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:19:18.845 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:18.845 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:19:18.845 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:18.845 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1368 (0x7f431bfe9c10) [ CSSD]2013-05-23 16:19:18.845 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9c10 (RPC#1368) tag(558002c) sent to node 1 [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0558002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014493, gin 1, member 0 [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082530/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9c10 (RPC#1368) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9c10, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:19:18.846 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1368 [ CSSD]2013-05-23 16:19:18.852 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:18.852 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:18.852 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:18.852 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1369 (0x7f431bfe9c90) [ CSSD]2013-05-23 16:19:18.853 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9c90 (RPC#1369) tag(559002c) sent to node 1 [ CSSD]2013-05-23 16:19:18.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0559002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:18.853 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:18.853 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9c90 (RPC#1369) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:18.853 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9c90, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:19:18.853 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1369 [ CSSD]2013-05-23 16:19:18.853 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:19:19.056 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091490 [ CSSD]2013-05-23 16:19:19.056 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091490) con(0x7f431c0e6720) skgpid ospid 9273 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:19:20.659 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:19:20.659 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1370 (0x7f431bfe9d10) [ CSSD]2013-05-23 16:19:20.660 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9d10 (RPC#1370) tag(55a002c) sent to node 1 [ CSSD]2013-05-23 16:19:20.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:20.660 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1370 (0x7f431bfe9d10) [ CSSD]2013-05-23 16:19:20.660 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9d10 (RPC#1370) state 6, flags 0x202 [ CSSD]2013-05-23 16:19:20.660 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9d10, ret 0, client (nil) [ CSSD]2013-05-23 16:19:20.660 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1370 [ CSSD]2013-05-23 16:19:20.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:20.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:20.804 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:22.419 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:22.419 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:22.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:22.810 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:22.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:23.323 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091490) allocated [ CSSD]2013-05-23 16:19:23.324 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:19:23.324 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091490 - 1,2,3 [ CSSD]2013-05-23 16:19:23.324 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c091490) pid(9310) version 11:1:1:4 [ CSSD]2013-05-23 16:19:23.330 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c091490), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:19:23.330 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:23.330 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f570, requested num -1 [ CSSD]2013-05-23 16:19:23.330 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:23.330 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1371 (0x7f431bfe9d90) [ CSSD]2013-05-23 16:19:23.330 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9d90 (RPC#1371) tag(55b002c) sent to node 1 [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(055b002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014494, gin 1, member 0 [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9d90 (RPC#1371) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9d90, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:19:23.331 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1371 [ CSSD]2013-05-23 16:19:23.338 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:23.338 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:23.338 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f360) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:23.338 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1372 (0x7f431bfe9e10) [ CSSD]2013-05-23 16:19:23.338 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9e10 (RPC#1372) tag(55c002c) sent to node 1 [ CSSD]2013-05-23 16:19:23.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(055c002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:23.338 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:23.338 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9e10 (RPC#1372) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:23.338 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9e10, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:19:23.338 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1372 [ CSSD]2013-05-23 16:19:23.338 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f360) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:19:23.541 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091490 [ CSSD]2013-05-23 16:19:23.541 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091490) con(0x7f431c108e60) skgpid ospid 9310 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:19:24.858 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:24.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:24.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:25.071 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:19:25.072 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:19:25.072 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:19:25.072 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134c40) pid(9364/9364) version 11:1:1:4 [ CSSD]2013-05-23 16:19:25.077 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:19:25.077 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:25.077 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:19:25.077 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:19:25.078 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:19:25.078 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:19:25.080 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:19:25.080 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:19:25.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:25.163 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014495, gin 1, member 0 [ CSSD]2013-05-23 16:19:25.163 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f75a0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:19:25.163 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:19:25.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:25.169 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:19:25.169 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:19:26.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:26.867 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:26.884 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:27.422 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:27.422 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:28.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:28.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:28.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:30.682 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:19:30.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1373 (0x7f431bfe9e90) [ CSSD]2013-05-23 16:19:30.682 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9e90 (RPC#1373) tag(55d002c) sent to node 1 [ CSSD]2013-05-23 16:19:30.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:30.683 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1373 (0x7f431bfe9e90) [ CSSD]2013-05-23 16:19:30.683 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9e90 (RPC#1373) state 6, flags 0x202 [ CSSD]2013-05-23 16:19:30.683 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe9e90, ret 0, client (nil) [ CSSD]2013-05-23 16:19:30.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1373 [ CSSD]2013-05-23 16:19:30.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:30.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:30.952 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:31.425 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:31.425 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:19:32.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:32.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:32.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:33.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:33.860 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014496, gin 1, member 0 [ CSSD]2013-05-23 16:19:33.860 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:19:33.860 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:19:33.866 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:33.866 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:19:33.866 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:19:34.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:34.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:34.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:36.428 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:36.429 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:36.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:36.968 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:36.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:38.578 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083840) allocated [ CSSD]2013-05-23 16:19:38.579 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:19:38.580 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083840 - 1,2,3 [ CSSD]2013-05-23 16:19:38.580 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829b0) proc(0x7f431c083840) pid(9491) version 11:1:1:4 [ CSSD]2013-05-23 16:19:38.597 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c083840), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:19:38.597 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:19:38.597 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:19:38.597 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:38.597 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1374 (0x7f431bfe9f10) [ CSSD]2013-05-23 16:19:38.597 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9f10 (RPC#1374) tag(55e002c) sent to node 1 [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(055e002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014497, gin 1, member 0 [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9f10 (RPC#1374) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe9f10, ret 0, client 0x7f431c137ac0 member 0x7f431c083720 [ CSSD]2013-05-23 16:19:38.597 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:19:38.598 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1374 [ CSSD]2013-05-23 16:19:38.604 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:19:38.604 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:38.604 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083720) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:38.604 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1375 (0x7f431bfe9f90) [ CSSD]2013-05-23 16:19:38.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe9f90 (RPC#1375) tag(55f002c) sent to node 1 [ CSSD]2013-05-23 16:19:38.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(055f002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:38.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:38.605 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe9f90 (RPC#1375) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:38.605 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe9f90, ret 0, client 0x7f431c137ac0 member 0x7f431c083720 [ CSSD]2013-05-23 16:19:38.605 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1375 [ CSSD]2013-05-23 16:19:38.605 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083720) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:19:38.807 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083840 [ CSSD]2013-05-23 16:19:38.807 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083840) con(0x7f431c0829b0) skgpid ospid 9491 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:19:38.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:38.980 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:39.027 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:40.710 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:19:40.710 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1376 (0x7f431bfea010) [ CSSD]2013-05-23 16:19:40.710 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea010 (RPC#1376) tag(560002c) sent to node 1 [ CSSD]2013-05-23 16:19:40.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:40.711 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1376 (0x7f431bfea010) [ CSSD]2013-05-23 16:19:40.711 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea010 (RPC#1376) state 6, flags 0x202 [ CSSD]2013-05-23 16:19:40.711 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea010, ret 0, client (nil) [ CSSD]2013-05-23 16:19:40.711 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1376 [ CSSD]2013-05-23 16:19:40.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:40.986 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:41.033 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:41.378 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:41.378 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:19:41.378 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:19:41.378 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:19:41.379 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:19:41.379 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c108e60) skgpid 9364 ospid 9364 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:19:41.432 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:41.432 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:42.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:42.992 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:43.045 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:45.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:45.003 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:45.051 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:46.436 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:46.436 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:47.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:47.009 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:47.077 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:49.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:49.061 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:49.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:49.119 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014498, gin 1, member 0 [ CSSD]2013-05-23 16:19:49.119 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:19:49.119 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:19:49.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:49.126 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:19:49.126 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:19:49.168 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:50.733 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:19:50.733 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1377 (0x7f431bfea090) [ CSSD]2013-05-23 16:19:50.734 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea090 (RPC#1377) tag(561002c) sent to node 1 [ CSSD]2013-05-23 16:19:50.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:50.734 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1377 (0x7f431bfea090) [ CSSD]2013-05-23 16:19:50.734 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea090 (RPC#1377) state 6, flags 0x202 [ CSSD]2013-05-23 16:19:50.734 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea090, ret 0, client (nil) [ CSSD]2013-05-23 16:19:50.734 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1377 [ CSSD]2013-05-23 16:19:51.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:51.087 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:51.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:51.440 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:51.440 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:53.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:53.093 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:53.225 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:53.845 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:19:53.846 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:19:53.847 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:19:53.847 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c134c40) pid(9653) version 11:1:1:4 [ CSSD]2013-05-23 16:19:53.853 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:19:53.853 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:53.853 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f570, requested num -1 [ CSSD]2013-05-23 16:19:53.853 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:53.853 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1378 (0x7f431bfea110) [ CSSD]2013-05-23 16:19:53.853 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea110 (RPC#1378) tag(562002c) sent to node 1 [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0562002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014499, gin 1, member 0 [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea110 (RPC#1378) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfea110, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:19:53.854 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1378 [ CSSD]2013-05-23 16:19:53.860 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:19:53.860 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:53.860 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f360) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:19:53.860 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1379 (0x7f431bfea190) [ CSSD]2013-05-23 16:19:53.860 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea190 (RPC#1379) tag(563002c) sent to node 1 [ CSSD]2013-05-23 16:19:53.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0563002c) incarnation(174631051) [ CSSD]2013-05-23 16:19:53.860 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:19:53.860 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea190 (RPC#1379) state 6, flags 0x100 [ CSSD]2013-05-23 16:19:53.860 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfea190, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:19:53.861 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1379 [ CSSD]2013-05-23 16:19:53.861 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f360) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:19:54.063 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:19:54.063 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c0e6720) skgpid ospid 9653 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:19:55.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:55.104 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:55.231 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:56.443 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:19:56.443 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:19:57.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:57.130 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:57.239 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:19:59.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:19:59.161 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:19:59.249 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:00.757 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:20:00.757 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1380 (0x7f431bfea210) [ CSSD]2013-05-23 16:20:00.757 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea210 (RPC#1380) tag(564002c) sent to node 1 [ CSSD]2013-05-23 16:20:00.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:00.758 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1380 (0x7f431bfea210) [ CSSD]2013-05-23 16:20:00.758 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea210 (RPC#1380) state 6, flags 0x202 [ CSSD]2013-05-23 16:20:00.758 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea210, ret 0, client (nil) [ CSSD]2013-05-23 16:20:00.758 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1380 [ CSSD]2013-05-23 16:20:01.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:01.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:01.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:01.447 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:01.447 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:03.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:03.172 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:03.264 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:04.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:04.374 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014500, gin 1, member 0 [ CSSD]2013-05-23 16:20:04.374 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:20:04.374 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:20:04.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:04.382 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:20:04.382 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:20:05.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:05.184 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:05.270 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:05.450 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:05.450 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:20:07.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:07.190 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:07.275 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:07.995 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:20:07.996 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:20:07.997 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:20:07.997 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c134c40) pid(10003) version 11:1:1:4 [ CSSD]2013-05-23 16:20:08.581 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:20:08.581 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c0e6720) skgpid ospid 10003 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:09.110 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091490) allocated [ CSSD]2013-05-23 16:20:09.112 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:20:09.112 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091490 - 1,2,3 [ CSSD]2013-05-23 16:20:09.112 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c091490) pid(10114) version 11:1:1:4 [ CSSD]2013-05-23 16:20:09.129 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c091490), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:20:09.129 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:09.129 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f570, requested num -1 [ CSSD]2013-05-23 16:20:09.129 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:09.129 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1381 (0x7f431bfea290) [ CSSD]2013-05-23 16:20:09.129 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea290 (RPC#1381) tag(565002c) sent to node 1 [ CSSD]2013-05-23 16:20:09.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0565002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:09.129 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014501, gin 1, member 0 [ CSSD]2013-05-23 16:20:09.129 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:20:09.130 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:20:09.130 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:20:09.130 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea290 (RPC#1381) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:09.130 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfea290, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:20:09.130 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:20:09.130 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1381 [ CSSD]2013-05-23 16:20:09.136 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:09.136 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:09.136 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f360) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:09.136 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1382 (0x7f431bfea310) [ CSSD]2013-05-23 16:20:09.137 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea310 (RPC#1382) tag(566002c) sent to node 1 [ CSSD]2013-05-23 16:20:09.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0566002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:09.137 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:09.137 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea310 (RPC#1382) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:09.137 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfea310, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:20:09.137 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1382 [ CSSD]2013-05-23 16:20:09.137 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f360) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:20:09.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:09.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:09.328 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:09.339 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091490 [ CSSD]2013-05-23 16:20:09.339 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091490) con(0x7f431c0e6720) skgpid ospid 10114 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:10.453 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:10.453 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:10.794 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:20:10.795 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1383 (0x7f431bfea390) [ CSSD]2013-05-23 16:20:10.795 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea390 (RPC#1383) tag(567002c) sent to node 1 [ CSSD]2013-05-23 16:20:10.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:10.796 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1383 (0x7f431bfea390) [ CSSD]2013-05-23 16:20:10.796 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea390 (RPC#1383) state 6, flags 0x202 [ CSSD]2013-05-23 16:20:10.796 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea390, ret 0, client (nil) [ CSSD]2013-05-23 16:20:10.796 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1383 [ CSSD]2013-05-23 16:20:11.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:11.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:11.335 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:13.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:13.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:13.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:15.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:15.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:15.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:15.285 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014502, gin 1, member 0 [ CSSD]2013-05-23 16:20:15.285 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:20:15.285 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:20:15.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:15.291 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:20:15.291 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfee0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:20:15.370 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:15.456 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:15.456 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:17.290 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:17.290 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:17.424 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:19.092 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091490) allocated [ CSSD]2013-05-23 16:20:19.093 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:20:19.094 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091490 - 1,2,3 [ CSSD]2013-05-23 16:20:19.094 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c091490) pid(10223) version 11:1:1:4 [ CSSD]2013-05-23 16:20:19.111 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c091490), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:20:19.111 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:19.111 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f570, requested num -1 [ CSSD]2013-05-23 16:20:19.111 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:19.111 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1384 (0x7f431bfea410) [ CSSD]2013-05-23 16:20:19.111 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea410 (RPC#1384) tag(568002c) sent to node 1 [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0568002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014503, gin 1, member 0 [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea410 (RPC#1384) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfea410, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:20:19.111 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:20:19.112 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1384 [ CSSD]2013-05-23 16:20:19.119 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:19.119 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:19.119 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f360) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:19.119 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1385 (0x7f431bfea490) [ CSSD]2013-05-23 16:20:19.119 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea490 (RPC#1385) tag(569002c) sent to node 1 [ CSSD]2013-05-23 16:20:19.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0569002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:19.120 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:19.120 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea490 (RPC#1385) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:19.120 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfea490, ret 0, client 0x7f431c121510 member 0x7f431c13f360 [ CSSD]2013-05-23 16:20:19.120 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1385 [ CSSD]2013-05-23 16:20:19.120 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f360) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:20:19.136 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x205ed60) [ CSSD]2013-05-23 16:20:19.136 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x205ed60), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:20:19.136 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x205ed60), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:20:19.136 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 16:20:19.140 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:20:19.140 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2312950), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:20:19.140 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2312950), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:20:19.140 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:20:19.140 [1191196992] >TRACE: clssgmDeadProc: proc 0x231d6c0 [ CSSD]2013-05-23 16:20:19.141 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231d6c0) con(0x205ecb0) skgpid 4461 ospid 4461 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:19.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:19.302 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:19.322 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091490 [ CSSD]2013-05-23 16:20:19.323 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091490) con(0x7f431c0e6720) skgpid ospid 10223 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:19.430 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:19.634 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:19.634 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014504, gin 1, member 0 [ CSSD]2013-05-23 16:20:19.634 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:20:19.634 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:20:19.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:19.640 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:20:19.640 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:20:20.460 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:20.460 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:20.819 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:20:20.819 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1386 (0x7f431bfea510) [ CSSD]2013-05-23 16:20:20.820 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea510 (RPC#1386) tag(56a002c) sent to node 1 [ CSSD]2013-05-23 16:20:20.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:20.820 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1386 (0x7f431bfea510) [ CSSD]2013-05-23 16:20:20.820 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea510 (RPC#1386) state 6, flags 0x202 [ CSSD]2013-05-23 16:20:20.820 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea510, ret 0, client (nil) [ CSSD]2013-05-23 16:20:20.820 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1386 [ CSSD]2013-05-23 16:20:21.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:21.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:21.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:23.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:23.314 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:23.446 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:24.388 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091490) allocated [ CSSD]2013-05-23 16:20:24.390 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:20:24.391 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091490 - 1,2,3 [ CSSD]2013-05-23 16:20:24.391 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c091490) pid(10276) version 11:1:1:4 [ CSSD]2013-05-23 16:20:24.407 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c091490), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:20:24.407 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:24.407 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c13f560, requested num -1 [ CSSD]2013-05-23 16:20:24.407 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:24.407 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1387 (0x7f431bfea590) [ CSSD]2013-05-23 16:20:24.407 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea590 (RPC#1387) tag(56b002c) sent to node 1 [ CSSD]2013-05-23 16:20:24.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(056b002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:24.407 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014505, gin 1, member 0 [ CSSD]2013-05-23 16:20:24.407 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0825e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:20:24.408 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:20:24.408 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:20:24.408 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea590 (RPC#1387) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:24.408 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfea590, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:20:24.408 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:20:24.408 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1387 [ CSSD]2013-05-23 16:20:24.414 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:24.414 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:24.414 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f380) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:24.414 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1388 (0x7f431bfea610) [ CSSD]2013-05-23 16:20:24.414 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea610 (RPC#1388) tag(56c002c) sent to node 1 [ CSSD]2013-05-23 16:20:24.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(056c002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:24.415 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:24.415 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea610 (RPC#1388) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:24.415 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfea610, ret 0, client 0x7f431c121510 member 0x7f431c13f380 [ CSSD]2013-05-23 16:20:24.415 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1388 [ CSSD]2013-05-23 16:20:24.415 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f380) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:20:24.462 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:24.462 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:20:24.617 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091490 [ CSSD]2013-05-23 16:20:24.617 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091490) con(0x7f431c0e6720) skgpid ospid 10276 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:24.630 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134c40) allocated [ CSSD]2013-05-23 16:20:24.631 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:20:24.632 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134c40 - 1,2,3 [ CSSD]2013-05-23 16:20:24.632 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134c40) pid(9020/9020) version 11:1:1:4 [ CSSD]2013-05-23 16:20:24.648 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134c40), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:20:24.648 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:20:24.648 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:20:24.648 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:20:25.332 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:25.332 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:25.453 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:27.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:27.338 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:27.464 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:29.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:29.372 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:29.465 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:29.466 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:29.471 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:30.845 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:20:30.846 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1389 (0x7f431bfea690) [ CSSD]2013-05-23 16:20:30.846 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea690 (RPC#1389) tag(56d002c) sent to node 1 [ CSSD]2013-05-23 16:20:30.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:30.847 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1389 (0x7f431bfea690) [ CSSD]2013-05-23 16:20:30.847 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea690 (RPC#1389) state 6, flags 0x202 [ CSSD]2013-05-23 16:20:30.847 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea690, ret 0, client (nil) [ CSSD]2013-05-23 16:20:30.847 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1389 [ CSSD]2013-05-23 16:20:31.378 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:31.378 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:31.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:33.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:33.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:33.528 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:34.469 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:34.469 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:34.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:34.882 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014506, gin 1, member 0 [ CSSD]2013-05-23 16:20:34.882 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:20:34.882 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:20:34.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:34.889 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:20:34.889 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:20:35.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:35.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:35.534 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:37.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:37.419 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:37.545 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:39.472 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:39.473 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:39.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:39.473 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:39.591 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:39.650 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083860) allocated [ CSSD]2013-05-23 16:20:39.651 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:20:39.651 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083860 - 1,2,3 [ CSSD]2013-05-23 16:20:39.651 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083860) pid(10456) version 11:1:1:4 [ CSSD]2013-05-23 16:20:39.658 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c083860), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:20:39.658 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:20:39.658 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:20:39.658 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:39.658 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1390 (0x7f431bfea710) [ CSSD]2013-05-23 16:20:39.658 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea710 (RPC#1390) tag(56e002c) sent to node 1 [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(056e002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014507, gin 1, member 0 [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea710 (RPC#1390) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfea710, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:20:39.659 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1390 [ CSSD]2013-05-23 16:20:39.665 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:20:39.665 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:39.665 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:39.665 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1391 (0x7f431bfea790) [ CSSD]2013-05-23 16:20:39.665 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea790 (RPC#1391) tag(56f002c) sent to node 1 [ CSSD]2013-05-23 16:20:39.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(056f002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:39.665 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:39.665 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea790 (RPC#1391) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:39.665 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfea790, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:20:39.666 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1391 [ CSSD]2013-05-23 16:20:39.666 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:20:39.868 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083860 [ CSSD]2013-05-23 16:20:39.868 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083860) con(0x7f431c0829d0) skgpid ospid 10456 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:40.872 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:20:40.872 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1392 (0x7f431bfea810) [ CSSD]2013-05-23 16:20:40.872 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea810 (RPC#1392) tag(570002c) sent to node 1 [ CSSD]2013-05-23 16:20:40.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:40.873 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1392 (0x7f431bfea810) [ CSSD]2013-05-23 16:20:40.873 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea810 (RPC#1392) state 6, flags 0x202 [ CSSD]2013-05-23 16:20:40.873 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea810, ret 0, client (nil) [ CSSD]2013-05-23 16:20:40.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1392 [ CSSD]2013-05-23 16:20:41.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:41.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:41.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:43.476 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:43.476 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:20:43.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:43.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:43.598 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:45.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:45.514 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:45.623 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:47.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:47.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:47.635 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:48.479 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:48.479 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:49.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:49.592 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:49.642 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:50.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:50.142 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014508, gin 1, member 0 [ CSSD]2013-05-23 16:20:50.142 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:20:50.142 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:20:50.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:50.148 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:20:50.148 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:20:50.897 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:20:50.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1393 (0x7f431bfea890) [ CSSD]2013-05-23 16:20:50.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea890 (RPC#1393) tag(571002c) sent to node 1 [ CSSD]2013-05-23 16:20:50.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:50.898 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1393 (0x7f431bfea890) [ CSSD]2013-05-23 16:20:50.898 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea890 (RPC#1393) state 6, flags 0x202 [ CSSD]2013-05-23 16:20:50.898 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfea890, ret 0, client (nil) [ CSSD]2013-05-23 16:20:50.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1393 [ CSSD]2013-05-23 16:20:51.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:51.598 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:51.648 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:53.482 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:53.482 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:53.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:53.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:53.648 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:54.902 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083860) allocated [ CSSD]2013-05-23 16:20:54.903 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:20:54.903 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083860 - 1,2,3 [ CSSD]2013-05-23 16:20:54.903 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083860) pid(10597) version 11:1:1:4 [ CSSD]2013-05-23 16:20:54.920 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c083860), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:20:54.920 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:20:54.920 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:20:54.920 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:54.920 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1394 (0x7f431bfea910) [ CSSD]2013-05-23 16:20:54.920 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea910 (RPC#1394) tag(572002c) sent to node 1 [ CSSD]2013-05-23 16:20:54.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0572002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:54.920 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014509, gin 1, member 0 [ CSSD]2013-05-23 16:20:54.920 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:20:54.921 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:20:54.921 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:20:54.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea910 (RPC#1394) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:54.921 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfea910, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:20:54.921 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:20:54.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1394 [ CSSD]2013-05-23 16:20:54.928 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:20:54.928 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:54.928 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:54.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1395 (0x7f431bfea990) [ CSSD]2013-05-23 16:20:54.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfea990 (RPC#1395) tag(573002c) sent to node 1 [ CSSD]2013-05-23 16:20:54.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0573002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:54.928 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:54.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfea990 (RPC#1395) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:54.928 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfea990, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:20:54.928 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1395 [ CSSD]2013-05-23 16:20:54.928 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:20:55.131 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083860 [ CSSD]2013-05-23 16:20:55.131 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083860) con(0x7f431c0829d0) skgpid ospid 10597 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:55.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:55.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:55.675 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:56.637 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083860) allocated [ CSSD]2013-05-23 16:20:56.639 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:20:56.639 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083860 - 1,2,3 [ CSSD]2013-05-23 16:20:56.639 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083860) pid(10656) version 11:1:1:4 [ CSSD]2013-05-23 16:20:56.656 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c083860), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:20:56.656 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:20:56.656 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:20:56.656 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:56.656 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1396 (0x7f431bfeaa10) [ CSSD]2013-05-23 16:20:56.656 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeaa10 (RPC#1396) tag(574002c) sent to node 1 [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0574002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014510, gin 1, member 0 [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeaa10 (RPC#1396) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeaa10, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:20:56.657 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1396 [ CSSD]2013-05-23 16:20:56.663 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:20:56.663 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:56.663 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:20:56.663 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1397 (0x7f431bfeaa90) [ CSSD]2013-05-23 16:20:56.663 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeaa90 (RPC#1397) tag(575002c) sent to node 1 [ CSSD]2013-05-23 16:20:56.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0575002c) incarnation(174631051) [ CSSD]2013-05-23 16:20:56.663 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:20:56.663 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeaa90 (RPC#1397) state 6, flags 0x100 [ CSSD]2013-05-23 16:20:56.664 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeaa90, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:20:56.664 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1397 [ CSSD]2013-05-23 16:20:56.664 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:20:56.866 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083860 [ CSSD]2013-05-23 16:20:56.866 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083860) con(0x7f431c0829d0) skgpid ospid 10656 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:20:57.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:57.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:57.686 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:20:58.486 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:20:58.486 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:20:59.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:20:59.674 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:20:59.693 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:00.922 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:21:00.922 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1398 (0x7f431bfeab10) [ CSSD]2013-05-23 16:21:00.922 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeab10 (RPC#1398) tag(576002c) sent to node 1 [ CSSD]2013-05-23 16:21:00.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:00.923 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1398 (0x7f431bfeab10) [ CSSD]2013-05-23 16:21:00.923 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeab10 (RPC#1398) state 6, flags 0x202 [ CSSD]2013-05-23 16:21:00.923 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeab10, ret 0, client (nil) [ CSSD]2013-05-23 16:21:00.923 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1398 [ CSSD]2013-05-23 16:21:01.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:01.699 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:01.700 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:02.489 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:02.489 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:21:03.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:03.705 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:03.750 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:05.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:05.397 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014511, gin 1, member 0 [ CSSD]2013-05-23 16:21:05.397 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:21:05.397 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:21:05.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:05.403 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:21:05.403 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:21:05.716 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:05.716 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:05.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:07.492 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:07.492 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:07.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:07.721 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:07.847 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:09.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:09.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:09.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:10.159 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083860) allocated [ CSSD]2013-05-23 16:21:10.160 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:21:10.161 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083860 - 1,2,3 [ CSSD]2013-05-23 16:21:10.161 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083860) pid(10757) version 11:1:1:4 [ CSSD]2013-05-23 16:21:10.177 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c083860), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:21:10.177 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:21:10.177 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:21:10.177 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:10.177 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1399 (0x7f431bfeab90) [ CSSD]2013-05-23 16:21:10.177 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeab90 (RPC#1399) tag(577002c) sent to node 1 [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0577002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014512, gin 1, member 0 [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeab90 (RPC#1399) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeab90, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:21:10.178 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1399 [ CSSD]2013-05-23 16:21:10.185 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:21:10.185 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:10.185 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:10.185 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1400 (0x7f431bfeac10) [ CSSD]2013-05-23 16:21:10.185 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeac10 (RPC#1400) tag(578002c) sent to node 1 [ CSSD]2013-05-23 16:21:10.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0578002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:10.185 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:10.185 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeac10 (RPC#1400) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:10.185 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeac10, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:10.185 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1400 [ CSSD]2013-05-23 16:21:10.185 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:21:10.388 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083860 [ CSSD]2013-05-23 16:21:10.388 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083860) con(0x7f431c0829d0) skgpid ospid 10757 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:10.945 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:21:10.945 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1401 (0x7f431bfeac90) [ CSSD]2013-05-23 16:21:10.946 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeac90 (RPC#1401) tag(579002c) sent to node 1 [ CSSD]2013-05-23 16:21:10.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:10.946 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1401 (0x7f431bfeac90) [ CSSD]2013-05-23 16:21:10.946 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeac90 (RPC#1401) state 6, flags 0x202 [ CSSD]2013-05-23 16:21:10.946 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeac90, ret 0, client (nil) [ CSSD]2013-05-23 16:21:10.946 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1401 [ CSSD]2013-05-23 16:21:11.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:11.779 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:11.879 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:12.496 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:12.496 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:13.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:13.784 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:13.909 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:15.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:15.546 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014513, gin 1, member 0 [ CSSD]2013-05-23 16:21:15.546 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:21:15.546 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:21:15.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:15.551 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:21:15.551 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:21:15.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:15.797 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:15.916 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:17.280 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083860) allocated [ CSSD]2013-05-23 16:21:17.281 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:21:17.282 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083860 - 1,2,3 [ CSSD]2013-05-23 16:21:17.282 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083860) pid(10843) version 11:1:1:4 [ CSSD]2013-05-23 16:21:17.299 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c083860), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:21:17.299 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:21:17.299 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:21:17.299 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:21:17.299 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1402 (0x7f431bfead10) [ CSSD]2013-05-23 16:21:17.299 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfead10 (RPC#1402) tag(57a002c) sent to node 1 [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(057a002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014514, gin 1, member 0 [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfead10 (RPC#1402) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfead10, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:17.299 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:21:17.300 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1402 [ CSSD]2013-05-23 16:21:17.305 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:21:17.305 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:21:17.305 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:21:17.305 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1403 (0x7f431bfead90) [ CSSD]2013-05-23 16:21:17.305 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfead90 (RPC#1403) tag(57b002c) sent to node 1 [ CSSD]2013-05-23 16:21:17.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(057b002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:17.306 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:21:17.306 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfead90 (RPC#1403) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:17.306 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfead90, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:17.306 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1403 [ CSSD]2013-05-23 16:21:17.306 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:21:17.421 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1501f0) allocated [ CSSD]2013-05-23 16:21:17.423 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:21:17.423 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1501f0 - 1,2,3 [ CSSD]2013-05-23 16:21:17.423 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f6f0) proc(0x7f431c1501f0) pid(10848) version 11:1:1:4 [ CSSD]2013-05-23 16:21:17.438 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:21:17.440 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:21:17.442 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1501f0 [ CSSD]2013-05-23 16:21:17.442 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1501f0) con(0x7f431c13f6f0) skgpid ospid 10848 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:17.499 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:17.499 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:17.508 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083860 [ CSSD]2013-05-23 16:21:17.508 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083860) con(0x7f431c0829d0) skgpid ospid 10843 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:17.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:17.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:17.948 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:19.361 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083860) allocated [ CSSD]2013-05-23 16:21:19.362 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:21:19.363 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083860 - 1,2,3 [ CSSD]2013-05-23 16:21:19.363 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083860) pid(10870) version 11:1:1:4 [ CSSD]2013-05-23 16:21:19.380 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c083860), client(1/0x7f431c137ac0) [ CSSD]2013-05-23 16:21:19.380 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:21:19.380 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ac0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:21:19.380 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:19.380 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1404 (0x7f431bfeae10) [ CSSD]2013-05-23 16:21:19.380 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeae10 (RPC#1404) tag(57c002c) sent to node 1 [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(057c002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014515, gin 1, member 0 [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeae10 (RPC#1404) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeae10, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ac0 [ CSSD]2013-05-23 16:21:19.381 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1404 [ CSSD]2013-05-23 16:21:19.387 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ac0) [ CSSD]2013-05-23 16:21:19.387 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ac0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:19.387 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:19.387 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1405 (0x7f431bfeae90) [ CSSD]2013-05-23 16:21:19.387 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeae90 (RPC#1405) tag(57d002c) sent to node 1 [ CSSD]2013-05-23 16:21:19.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(057d002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:19.388 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:19.388 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeae90 (RPC#1405) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:19.388 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeae90, ret 0, client 0x7f431c137ac0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:19.388 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1405 [ CSSD]2013-05-23 16:21:19.388 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:21:19.590 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083860 [ CSSD]2013-05-23 16:21:19.590 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083860) con(0x7f431c0829d0) skgpid ospid 10870 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:19.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:19.875 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:19.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:20.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:20.653 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014516, gin 1, member 0 [ CSSD]2013-05-23 16:21:20.653 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:21:20.653 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:21:20.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:20.659 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:21:20.659 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:21:20.968 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:21:20.968 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1406 (0x7f431bfeaf10) [ CSSD]2013-05-23 16:21:20.968 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeaf10 (RPC#1406) tag(57e002c) sent to node 1 [ CSSD]2013-05-23 16:21:20.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:20.969 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1406 (0x7f431bfeaf10) [ CSSD]2013-05-23 16:21:20.969 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeaf10 (RPC#1406) state 6, flags 0x202 [ CSSD]2013-05-23 16:21:20.969 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeaf10, ret 0, client (nil) [ CSSD]2013-05-23 16:21:20.969 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1406 [ CSSD]2013-05-23 16:21:21.502 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:21.502 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:21:21.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:21.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:21.961 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:23.427 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f1c80) [ CSSD]2013-05-23 16:21:23.427 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f1c80), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:21:23.427 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f1c80), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:21:23.427 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 27 [ CSSD]2013-05-23 16:21:23.437 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23029d0) [ CSSD]2013-05-23 16:21:23.437 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23029d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:21:23.437 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23029d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:21:23.437 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:21:23.437 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:21:23.438 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid 5097 ospid 5097 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:23.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:23.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:23.971 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:25.427 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083850) allocated [ CSSD]2013-05-23 16:21:25.429 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:21:25.429 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083850 - 1,2,3 [ CSSD]2013-05-23 16:21:25.429 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083850) pid(10935) version 11:1:1:4 [ CSSD]2013-05-23 16:21:25.436 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c083850), client(1/0x7f431c137ab0) [ CSSD]2013-05-23 16:21:25.436 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ab0) [ CSSD]2013-05-23 16:21:25.436 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ab0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:21:25.436 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:25.436 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1407 (0x7f431bfeaf90) [ CSSD]2013-05-23 16:21:25.436 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeaf90 (RPC#1407) tag(57f002c) sent to node 1 [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(057f002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014517, gin 1, member 0 [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeaf90 (RPC#1407) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeaf90, ret 0, client 0x7f431c137ab0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:25.436 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ab0 [ CSSD]2013-05-23 16:21:25.437 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1407 [ CSSD]2013-05-23 16:21:25.443 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ab0) [ CSSD]2013-05-23 16:21:25.443 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ab0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:25.443 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:25.443 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1408 (0x7f431bfeb010) [ CSSD]2013-05-23 16:21:25.443 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb010 (RPC#1408) tag(580002c) sent to node 1 [ CSSD]2013-05-23 16:21:25.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0580002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:25.444 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:25.444 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb010 (RPC#1408) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:25.444 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb010, ret 0, client 0x7f431c137ab0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:25.444 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1408 [ CSSD]2013-05-23 16:21:25.444 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:21:25.646 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083850 [ CSSD]2013-05-23 16:21:25.646 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083850) con(0x7f431c0829d0) skgpid ospid 10935 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:25.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:25.917 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:25.977 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:26.506 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:26.506 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:27.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:27.942 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:27.987 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:29.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:29.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:30.012 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:30.992 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:21:30.992 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1409 (0x7f431bfeb090) [ CSSD]2013-05-23 16:21:30.992 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb090 (RPC#1409) tag(581002c) sent to node 1 [ CSSD]2013-05-23 16:21:30.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:30.993 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1409 (0x7f431bfeb090) [ CSSD]2013-05-23 16:21:30.993 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb090 (RPC#1409) state 6, flags 0x202 [ CSSD]2013-05-23 16:21:30.993 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeb090, ret 0, client (nil) [ CSSD]2013-05-23 16:21:30.993 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1409 [ CSSD]2013-05-23 16:21:31.509 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:31.510 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:31.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:31.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:32.019 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:33.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:33.983 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:34.051 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:35.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:35.901 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014518, gin 1, member 0 [ CSSD]2013-05-23 16:21:35.902 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:21:35.902 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:21:35.907 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:35.907 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:21:35.907 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:21:35.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:35.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:36.058 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:36.513 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:36.514 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:37.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:37.995 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:38.068 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:40.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:40.034 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:40.074 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:40.679 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083850) allocated [ CSSD]2013-05-23 16:21:40.680 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:21:40.680 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083850 - 1,2,3 [ CSSD]2013-05-23 16:21:40.680 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c083850) pid(11090) version 11:1:1:4 [ CSSD]2013-05-23 16:21:40.687 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c083850), client(1/0x7f431c137ab0) [ CSSD]2013-05-23 16:21:40.687 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ab0) [ CSSD]2013-05-23 16:21:40.687 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ab0 with con 0x7f431c1375b0, requested num -1 [ CSSD]2013-05-23 16:21:40.687 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:40.687 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1410 (0x7f431bfeb110) [ CSSD]2013-05-23 16:21:40.687 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb110 (RPC#1410) tag(582002c) sent to node 1 [ CSSD]2013-05-23 16:21:40.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0582002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:40.687 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014519, gin 1, member 0 [ CSSD]2013-05-23 16:21:40.687 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e340/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:21:40.688 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:21:40.688 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:21:40.688 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb110 (RPC#1410) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:40.688 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb110, ret 0, client 0x7f431c137ab0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:40.688 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ab0 [ CSSD]2013-05-23 16:21:40.688 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1410 [ CSSD]2013-05-23 16:21:40.694 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ab0) [ CSSD]2013-05-23 16:21:40.694 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ab0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:40.694 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:40.694 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1411 (0x7f431bfeb190) [ CSSD]2013-05-23 16:21:40.694 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb190 (RPC#1411) tag(583002c) sent to node 1 [ CSSD]2013-05-23 16:21:40.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0583002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:40.695 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:40.695 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb190 (RPC#1411) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:40.695 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb190, ret 0, client 0x7f431c137ab0 member 0x7f431c083740 [ CSSD]2013-05-23 16:21:40.695 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1411 [ CSSD]2013-05-23 16:21:40.695 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:21:40.897 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083850 [ CSSD]2013-05-23 16:21:40.897 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083850) con(0x7f431c0829d0) skgpid ospid 11090 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:41.016 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:21:41.016 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1412 (0x7f431bfeb210) [ CSSD]2013-05-23 16:21:41.017 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb210 (RPC#1412) tag(584002c) sent to node 1 [ CSSD]2013-05-23 16:21:41.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:41.017 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1412 (0x7f431bfeb210) [ CSSD]2013-05-23 16:21:41.017 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb210 (RPC#1412) state 6, flags 0x202 [ CSSD]2013-05-23 16:21:41.017 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeb210, ret 0, client (nil) [ CSSD]2013-05-23 16:21:41.017 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1412 [ CSSD]2013-05-23 16:21:41.517 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:41.517 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:42.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:42.040 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:42.081 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:42.309 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838b0) allocated [ CSSD]2013-05-23 16:21:42.309 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:21:42.310 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838b0 - 1,2,3 [ CSSD]2013-05-23 16:21:42.310 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0829d0) proc(0x7f431c0838b0) pid(10926/10926) version 11:1:1:4 [ CSSD]2013-05-23 16:21:42.326 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c0838b0), client(1/0x7f431c1377b0) [ CSSD]2013-05-23 16:21:42.327 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c1377b0) [ CSSD]2013-05-23 16:21:42.327 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:21:42.327 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:21:44.091 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:44.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:44.094 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:46.097 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:46.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:46.119 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:46.520 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:46.520 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:48.125 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:48.125 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:48.149 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:50.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:50.136 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:50.174 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:51.043 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:21:51.043 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1413 (0x7f431bfeb290) [ CSSD]2013-05-23 16:21:51.044 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb290 (RPC#1413) tag(585002c) sent to node 1 [ CSSD]2013-05-23 16:21:51.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:51.044 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1413 (0x7f431bfeb290) [ CSSD]2013-05-23 16:21:51.044 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb290 (RPC#1413) state 6, flags 0x202 [ CSSD]2013-05-23 16:21:51.044 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeb290, ret 0, client (nil) [ CSSD]2013-05-23 16:21:51.044 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1413 [ CSSD]2013-05-23 16:21:51.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:51.150 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014520, gin 1, member 0 [ CSSD]2013-05-23 16:21:51.150 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:21:51.150 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:21:51.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:51.156 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:21:51.156 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:21:51.523 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:51.524 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:21:52.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:52.142 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:52.180 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:54.192 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:54.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:54.194 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:55.526 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:21:55.526 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:21:55.929 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c150510) allocated [ CSSD]2013-05-23 16:21:55.930 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:21:55.930 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c150510 - 1,2,3 [ CSSD]2013-05-23 16:21:55.930 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137920) proc(0x7f431c150510) pid(11286) version 11:1:1:4 [ CSSD]2013-05-23 16:21:55.936 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c150510), client(1/0x7f431c150920) [ CSSD]2013-05-23 16:21:55.936 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c150920) [ CSSD]2013-05-23 16:21:55.936 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c150920 with con 0x7f431c1507a0, requested num -1 [ CSSD]2013-05-23 16:21:55.936 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:55.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1414 (0x7f431bfeb310) [ CSSD]2013-05-23 16:21:55.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb310 (RPC#1414) tag(586002c) sent to node 1 [ CSSD]2013-05-23 16:21:55.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0586002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014521, gin 1, member 0 [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c148240/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb310 (RPC#1414) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb310, ret 0, client 0x7f431c150920 member 0x7f431c1503f0 [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c150920 [ CSSD]2013-05-23 16:21:55.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1414 [ CSSD]2013-05-23 16:21:55.943 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c150920) [ CSSD]2013-05-23 16:21:55.943 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c150920), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:55.943 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1503f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:21:55.943 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1415 (0x7f431bfeb390) [ CSSD]2013-05-23 16:21:55.943 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb390 (RPC#1415) tag(587002c) sent to node 1 [ CSSD]2013-05-23 16:21:55.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0587002c) incarnation(174631051) [ CSSD]2013-05-23 16:21:55.944 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:21:55.944 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb390 (RPC#1415) state 6, flags 0x100 [ CSSD]2013-05-23 16:21:55.944 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb390, ret 0, client 0x7f431c150920 member 0x7f431c1503f0 [ CSSD]2013-05-23 16:21:55.944 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1415 [ CSSD]2013-05-23 16:21:55.944 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1503f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:21:56.147 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c150510 [ CSSD]2013-05-23 16:21:56.147 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c150510) con(0x7f431c137920) skgpid ospid 11286 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:21:56.198 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:21:56.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:56.219 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:58.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:21:58.224 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:21:58.250 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:00.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:00.237 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:00.257 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:00.530 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:00.530 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:01.068 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:22:01.069 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1416 (0x7f431bfeb410) [ CSSD]2013-05-23 16:22:01.069 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb410 (RPC#1416) tag(588002c) sent to node 1 [ CSSD]2013-05-23 16:22:01.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:01.069 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1416 (0x7f431bfeb410) [ CSSD]2013-05-23 16:22:01.069 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb410 (RPC#1416) state 6, flags 0x202 [ CSSD]2013-05-23 16:22:01.069 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeb410, ret 0, client (nil) [ CSSD]2013-05-23 16:22:01.069 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1416 [ CSSD]2013-05-23 16:22:02.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:02.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:02.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:04.291 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:04.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:04.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:04.366 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c0838b0), client(2/0x7f431c137bd0) [ CSSD]2013-05-23 16:22:04.366 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c137bd0) [ CSSD]2013-05-23 16:22:04.366 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:22:04.366 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 28 [ CSSD]2013-05-23 16:22:05.533 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:05.533 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:06.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:06.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:06.300 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:06.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:06.400 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014522, gin 1, member 0 [ CSSD]2013-05-23 16:22:06.400 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:22:06.400 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:22:06.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:06.406 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:22:06.406 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:22:06.682 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1507c0) allocated [ CSSD]2013-05-23 16:22:06.683 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:22:06.684 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1507c0 - 1,2,3 [ CSSD]2013-05-23 16:22:06.684 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d40) proc(0x7f431c1507c0) pid(11398) version 11:1:1:4 [ CSSD]2013-05-23 16:22:06.700 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1507c0), client(1/0x7f431c148470) [ CSSD]2013-05-23 16:22:06.701 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c148470) [ CSSD]2013-05-23 16:22:06.701 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c148470 with con 0x7f431c147f60, requested num -1 [ CSSD]2013-05-23 16:22:06.701 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:06.701 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1417 (0x7f431bfeb490) [ CSSD]2013-05-23 16:22:06.701 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb490 (RPC#1417) tag(589002c) sent to node 1 [ CSSD]2013-05-23 16:22:06.701 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0589002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:06.701 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014523, gin 1, member 0 [ CSSD]2013-05-23 16:22:06.701 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1485e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:22:06.701 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:06.701 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:22:06.702 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb490 (RPC#1417) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:06.702 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb490, ret 0, client 0x7f431c148470 member 0x7f431c1488e0 [ CSSD]2013-05-23 16:22:06.702 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c148470 [ CSSD]2013-05-23 16:22:06.702 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1417 [ CSSD]2013-05-23 16:22:06.709 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c148470) [ CSSD]2013-05-23 16:22:06.709 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c148470), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:06.709 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1488e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:06.709 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1418 (0x7f431bfeb510) [ CSSD]2013-05-23 16:22:06.709 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb510 (RPC#1418) tag(58a002c) sent to node 1 [ CSSD]2013-05-23 16:22:06.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(058a002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:06.710 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:06.710 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb510 (RPC#1418) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:06.710 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb510, ret 0, client 0x7f431c148470 member 0x7f431c1488e0 [ CSSD]2013-05-23 16:22:06.710 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1418 [ CSSD]2013-05-23 16:22:06.710 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1488e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:06.912 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1507c0 [ CSSD]2013-05-23 16:22:06.912 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1507c0) con(0x7f431c137d40) skgpid ospid 11398 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:08.300 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:08.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:08.306 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:10.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:10.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:10.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:10.536 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:10.537 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:11.100 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:22:11.100 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1419 (0x7f431bfeb590) [ CSSD]2013-05-23 16:22:11.100 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb590 (RPC#1419) tag(58b002c) sent to node 1 [ CSSD]2013-05-23 16:22:11.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:11.101 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1419 (0x7f431bfeb590) [ CSSD]2013-05-23 16:22:11.101 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb590 (RPC#1419) state 6, flags 0x202 [ CSSD]2013-05-23 16:22:11.101 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeb590, ret 0, client (nil) [ CSSD]2013-05-23 16:22:11.101 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1419 [ CSSD]2013-05-23 16:22:11.185 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147de0) allocated [ CSSD]2013-05-23 16:22:11.186 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:22:11.186 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147de0 - 1,2,3 [ CSSD]2013-05-23 16:22:11.186 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14e720) proc(0x7f431c147de0) pid(11447) version 11:1:1:4 [ CSSD]2013-05-23 16:22:11.203 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c147de0), client(1/0x7f431c148580) [ CSSD]2013-05-23 16:22:11.203 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c148580) [ CSSD]2013-05-23 16:22:11.203 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c148580 with con 0x7f431c148070, requested num -1 [ CSSD]2013-05-23 16:22:11.203 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:11.203 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1420 (0x7f431bfeb610) [ CSSD]2013-05-23 16:22:11.203 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb610 (RPC#1420) tag(58c002c) sent to node 1 [ CSSD]2013-05-23 16:22:11.203 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(058c002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:11.203 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014524, gin 1, member 0 [ CSSD]2013-05-23 16:22:11.203 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1486f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:22:11.204 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:11.204 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:22:11.204 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb610 (RPC#1420) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:11.204 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb610, ret 0, client 0x7f431c148580 member 0x7f431c150880 [ CSSD]2013-05-23 16:22:11.204 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c148580 [ CSSD]2013-05-23 16:22:11.204 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1420 [ CSSD]2013-05-23 16:22:11.210 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c148580) [ CSSD]2013-05-23 16:22:11.210 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c148580), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:11.210 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c150880) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:11.210 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1421 (0x7f431bfeb690) [ CSSD]2013-05-23 16:22:11.210 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb690 (RPC#1421) tag(58d002c) sent to node 1 [ CSSD]2013-05-23 16:22:11.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(058d002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:11.210 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:11.210 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb690 (RPC#1421) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:11.210 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb690, ret 0, client 0x7f431c148580 member 0x7f431c150880 [ CSSD]2013-05-23 16:22:11.210 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1421 [ CSSD]2013-05-23 16:22:11.210 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c150880) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:11.413 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147de0 [ CSSD]2013-05-23 16:22:11.413 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147de0) con(0x7f431c14e720) skgpid ospid 11447 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:12.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:12.324 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:12.332 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:14.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:14.335 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:14.381 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:14.539 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:14.539 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:22:15.770 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147ff0) allocated [ CSSD]2013-05-23 16:22:15.771 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:22:15.772 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147ff0 - 1,2,3 [ CSSD]2013-05-23 16:22:15.772 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14e8d0) proc(0x7f431c147ff0) pid(11712) version 11:1:1:4 [ CSSD]2013-05-23 16:22:15.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:15.802 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014525, gin 1, member 0 [ CSSD]2013-05-23 16:22:15.802 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:22:15.802 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:22:15.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:15.809 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:22:15.809 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:22:15.907 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c14a8c0) allocated [ CSSD]2013-05-23 16:22:15.909 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:22:15.910 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c14a8c0 - 1,2,3 [ CSSD]2013-05-23 16:22:15.910 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d40) proc(0x7f431c14a8c0) pid(11722) version 11:1:1:4 [ CSSD]2013-05-23 16:22:15.926 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:22:15.928 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:22:15.931 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c14a8c0 [ CSSD]2013-05-23 16:22:15.931 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c14a8c0) con(0x7f431c137d40) skgpid ospid 11722 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:15.998 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(67/0x7f431c14af40) [ CSSD]2013-05-23 16:22:15.998 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 67 (0x7f431c14af40) [ CSSD]2013-05-23 16:22:15.998 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c14af40 with con 0x7f431c137d40, requested num -1 [ CSSD]2013-05-23 16:22:15.998 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:22:15.998 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1422 (0x7f431bfeb710) [ CSSD]2013-05-23 16:22:15.998 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb710 (RPC#1422) tag(58e002c) sent to node 1 [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(058e002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014526, gin 1, member 0 [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c14b0b0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb710 (RPC#1422) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb710, ret 0, client 0x7f431c14af40 member 0x7f431c147ed0 [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14af40 [ CSSD]2013-05-23 16:22:15.999 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1422 [ CSSD]2013-05-23 16:22:16.005 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 67 (0x7f431c14af40) [ CSSD]2013-05-23 16:22:16.005 [1191196992] >TRACE: clssgmExitGrock: client 67 (0x7f431c14af40), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:22:16.005 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c147ed0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:22:16.005 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1423 (0x7f431bfeb790) [ CSSD]2013-05-23 16:22:16.006 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb790 (RPC#1423) tag(58f002c) sent to node 1 [ CSSD]2013-05-23 16:22:16.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(058f002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:16.006 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:22:16.006 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb790 (RPC#1423) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:16.006 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb790, ret 0, client 0x7f431c14af40 member 0x7f431c147ed0 [ CSSD]2013-05-23 16:22:16.006 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1423 [ CSSD]2013-05-23 16:22:16.006 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c147ed0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:16.007 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147ff0 [ CSSD]2013-05-23 16:22:16.007 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147ff0) con(0x7f431c14e8d0) skgpid ospid 11712 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:16.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:16.341 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:16.388 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:18.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:18.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:18.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:19.543 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:19.543 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:19.632 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147ff0) allocated [ CSSD]2013-05-23 16:22:19.633 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:22:19.634 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147ff0 - 1,2,3 [ CSSD]2013-05-23 16:22:19.634 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14e8d0) proc(0x7f431c147ff0) pid(11757) version 11:1:1:4 [ CSSD]2013-05-23 16:22:19.640 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c147ff0), client(1/0x7f431c137e70) [ CSSD]2013-05-23 16:22:19.640 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 16:22:19.640 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137e70 with con 0x7f431c148250, requested num -1 [ CSSD]2013-05-23 16:22:19.640 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:19.640 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1424 (0x7f431bfeb810) [ CSSD]2013-05-23 16:22:19.640 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb810 (RPC#1424) tag(590002c) sent to node 1 [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0590002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014527, gin 1, member 0 [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c148830/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb810 (RPC#1424) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb810, ret 0, client 0x7f431c137e70 member 0x7f431c147ed0 [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137e70 [ CSSD]2013-05-23 16:22:19.641 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1424 [ CSSD]2013-05-23 16:22:19.647 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 16:22:19.647 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:19.647 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c147ed0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:19.647 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1425 (0x7f431bfeb890) [ CSSD]2013-05-23 16:22:19.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb890 (RPC#1425) tag(591002c) sent to node 1 [ CSSD]2013-05-23 16:22:19.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0591002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:19.647 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:19.647 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb890 (RPC#1425) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:19.647 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeb890, ret 0, client 0x7f431c137e70 member 0x7f431c147ed0 [ CSSD]2013-05-23 16:22:19.648 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1425 [ CSSD]2013-05-23 16:22:19.648 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c147ed0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:19.850 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147ff0 [ CSSD]2013-05-23 16:22:19.850 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147ff0) con(0x7f431c14e8d0) skgpid ospid 11757 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:20.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:20.398 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:20.407 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:21.128 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:22:21.128 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1426 (0x7f431bfeb910) [ CSSD]2013-05-23 16:22:21.129 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb910 (RPC#1426) tag(592002c) sent to node 1 [ CSSD]2013-05-23 16:22:21.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:21.129 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1426 (0x7f431bfeb910) [ CSSD]2013-05-23 16:22:21.129 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb910 (RPC#1426) state 6, flags 0x202 [ CSSD]2013-05-23 16:22:21.129 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeb910, ret 0, client (nil) [ CSSD]2013-05-23 16:22:21.129 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1426 [ CSSD]2013-05-23 16:22:21.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:21.653 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014528, gin 1, member 0 [ CSSD]2013-05-23 16:22:21.653 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:22:21.653 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:22:21.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:21.660 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:22:21.660 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:22:22.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:22.424 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:22.458 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:24.435 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:24.435 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:24.465 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:24.547 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:24.547 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:26.231 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c147ff0) allocated [ CSSD]2013-05-23 16:22:26.232 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:22:26.233 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c147ff0 - 1,2,3 [ CSSD]2013-05-23 16:22:26.233 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14e8d0) proc(0x7f431c147ff0) pid(10916/10916) version 11:1:1:4 [ CSSD]2013-05-23 16:22:26.238 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c147ff0), client(1/0x7f431c137e70) [ CSSD]2013-05-23 16:22:26.238 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 16:22:26.238 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:22:26.238 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:22:26.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:26.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:26.447 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c14af00) allocated [ CSSD]2013-05-23 16:22:26.448 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:22:26.449 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c14af00 - 1,2,3 [ CSSD]2013-05-23 16:22:26.449 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d40) proc(0x7f431c14af00) pid(11812) version 11:1:1:4 [ CSSD]2013-05-23 16:22:26.466 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c14af00), client(1/0x7f431c14b630) [ CSSD]2013-05-23 16:22:26.466 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14b630) [ CSSD]2013-05-23 16:22:26.466 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14b630 with con 0x7f431c14ae10, requested num -1 [ CSSD]2013-05-23 16:22:26.466 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:26.466 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1427 (0x7f431bfeb990) [ CSSD]2013-05-23 16:22:26.466 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeb990 (RPC#1427) tag(593002c) sent to node 1 [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0593002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014529, gin 1, member 0 [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14b7a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeb990 (RPC#1427) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeb990, ret 0, client 0x7f431c14b630 member 0x7f431c14baa0 [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14b630 [ CSSD]2013-05-23 16:22:26.467 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1427 [ CSSD]2013-05-23 16:22:26.474 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14b630) [ CSSD]2013-05-23 16:22:26.474 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14b630), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:26.474 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c14baa0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:26.474 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1428 (0x7f431bfeba10) [ CSSD]2013-05-23 16:22:26.475 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeba10 (RPC#1428) tag(594002c) sent to node 1 [ CSSD]2013-05-23 16:22:26.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0594002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:26.475 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:26.475 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeba10 (RPC#1428) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:26.475 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeba10, ret 0, client 0x7f431c14b630 member 0x7f431c14baa0 [ CSSD]2013-05-23 16:22:26.475 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1428 [ CSSD]2013-05-23 16:22:26.475 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c14baa0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:26.490 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:26.678 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c14af00 [ CSSD]2013-05-23 16:22:26.678 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c14af00) con(0x7f431c137d40) skgpid ospid 11812 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:28.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:28.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:28.521 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:29.551 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:29.551 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:30.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:30.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:30.527 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:31.157 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:22:31.158 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1429 (0x7f431bfeba90) [ CSSD]2013-05-23 16:22:31.158 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeba90 (RPC#1429) tag(595002c) sent to node 1 [ CSSD]2013-05-23 16:22:31.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:31.158 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1429 (0x7f431bfeba90) [ CSSD]2013-05-23 16:22:31.158 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeba90 (RPC#1429) state 6, flags 0x202 [ CSSD]2013-05-23 16:22:31.158 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeba90, ret 0, client (nil) [ CSSD]2013-05-23 16:22:31.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1429 [ CSSD]2013-05-23 16:22:32.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:32.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:32.558 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:34.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:34.535 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:34.554 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:34.555 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:34.565 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:36.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:36.541 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:36.571 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:36.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:36.913 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014530, gin 1, member 0 [ CSSD]2013-05-23 16:22:36.913 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:22:36.913 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:22:36.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:36.953 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:22:36.953 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:22:38.546 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:38.546 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:38.561 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:39.558 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:39.558 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:40.587 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:40.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:40.599 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:41.180 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:22:41.180 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1430 (0x7f431bfebb10) [ CSSD]2013-05-23 16:22:41.181 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebb10 (RPC#1430) tag(596002c) sent to node 1 [ CSSD]2013-05-23 16:22:41.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:41.181 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1430 (0x7f431bfebb10) [ CSSD]2013-05-23 16:22:41.181 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebb10 (RPC#1430) state 6, flags 0x202 [ CSSD]2013-05-23 16:22:41.181 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfebb10, ret 0, client (nil) [ CSSD]2013-05-23 16:22:41.181 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1430 [ CSSD]2013-05-23 16:22:41.727 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c14ae60) allocated [ CSSD]2013-05-23 16:22:41.728 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:22:41.729 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c14ae60 - 1,2,3 [ CSSD]2013-05-23 16:22:41.729 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d40) proc(0x7f431c14ae60) pid(11954) version 11:1:1:4 [ CSSD]2013-05-23 16:22:41.746 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c14ae60), client(1/0x7f431c14b660) [ CSSD]2013-05-23 16:22:41.746 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14b660) [ CSSD]2013-05-23 16:22:41.746 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14b660 with con 0x7f431c14b150, requested num -1 [ CSSD]2013-05-23 16:22:41.746 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:41.746 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1431 (0x7f431bfebb90) [ CSSD]2013-05-23 16:22:41.746 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebb90 (RPC#1431) tag(597002c) sent to node 1 [ CSSD]2013-05-23 16:22:41.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0597002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:41.746 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014531, gin 1, member 0 [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14b7d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebb90 (RPC#1431) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfebb90, ret 0, client 0x7f431c14b660 member 0x7f431c14ad40 [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14b660 [ CSSD]2013-05-23 16:22:41.747 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1431 [ CSSD]2013-05-23 16:22:41.753 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14b660) [ CSSD]2013-05-23 16:22:41.753 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14b660), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:41.753 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c14ad40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:41.753 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1432 (0x7f431bfebc10) [ CSSD]2013-05-23 16:22:41.753 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebc10 (RPC#1432) tag(598002c) sent to node 1 [ CSSD]2013-05-23 16:22:41.753 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0598002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:41.753 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:41.753 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebc10 (RPC#1432) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:41.753 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfebc10, ret 0, client 0x7f431c14b660 member 0x7f431c14ad40 [ CSSD]2013-05-23 16:22:41.753 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1432 [ CSSD]2013-05-23 16:22:41.753 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c14ad40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:41.956 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c14ae60 [ CSSD]2013-05-23 16:22:41.956 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c14ae60) con(0x7f431c137d40) skgpid ospid 11954 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:42.599 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:42.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:42.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:44.562 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:44.562 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:44.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:44.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:44.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:46.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:46.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:46.621 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:48.565 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:48.565 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:22:48.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:48.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:48.663 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:49.154 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c150ab0) [ CSSD]2013-05-23 16:22:49.154 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c150ab0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:22:49.154 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c150ab0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:22:49.154 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 27 [ CSSD]2013-05-23 16:22:49.157 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c134760) [ CSSD]2013-05-23 16:22:49.157 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c134760), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:22:49.157 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c134760), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:22:49.157 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:22:49.159 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c134470) [ CSSD]2013-05-23 16:22:49.159 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c134470), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:22:49.159 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c134470), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:22:49.159 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:22:49.161 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12ba00) [ CSSD]2013-05-23 16:22:49.161 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12ba00), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:22:49.161 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12ba00), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:22:49.162 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:22:49.162 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147cb0 [ CSSD]2013-05-23 16:22:49.162 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147cb0) con(0x7f431c0e6e20) skgpid 6343 ospid 6343 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:49.168 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:22:49.168 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:22:49.168 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:22:49.168 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:22:49.168 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133a90 [ CSSD]2013-05-23 16:22:49.168 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133a90) con(0x7f431c12b850) skgpid 6335 ospid 6335 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:49.169 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:22:49.169 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:22:49.169 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c10a560), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:22:49.169 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:22:49.169 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 16:22:49.169 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c0e6b70) skgpid 6345 ospid 6345 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:50.669 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:50.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:50.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:51.203 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:22:51.204 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1433 (0x7f431bfebc90) [ CSSD]2013-05-23 16:22:51.204 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebc90 (RPC#1433) tag(599002c) sent to node 1 [ CSSD]2013-05-23 16:22:51.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:51.204 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1433 (0x7f431bfebc90) [ CSSD]2013-05-23 16:22:51.204 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebc90 (RPC#1433) state 6, flags 0x202 [ CSSD]2013-05-23 16:22:51.204 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfebc90, ret 0, client (nil) [ CSSD]2013-05-23 16:22:51.205 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1433 [ CSSD]2013-05-23 16:22:52.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:52.166 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014532, gin 1, member 0 [ CSSD]2013-05-23 16:22:52.166 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:22:52.166 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:22:52.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:52.172 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:22:52.172 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:22:52.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:52.706 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:52.706 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:53.568 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:53.569 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:54.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:54.717 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:54.725 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:56.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:56.743 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:56.750 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:22:56.993 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 16:22:56.994 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:22:56.995 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 16:22:56.995 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c12f520) pid(12184) version 11:1:1:4 [ CSSD]2013-05-23 16:22:57.001 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12f520), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:22:57.001 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:22:57.001 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c137d40, requested num -1 [ CSSD]2013-05-23 16:22:57.001 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:57.001 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1434 (0x7f431bfebd10) [ CSSD]2013-05-23 16:22:57.001 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebd10 (RPC#1434) tag(59a002c) sent to node 1 [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(059a002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014533, gin 1, member 0 [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebd10 (RPC#1434) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfebd10, ret 0, client 0x7f431c10a560 member 0x7f431c0e69d0 [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:22:57.002 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1434 [ CSSD]2013-05-23 16:22:57.009 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:22:57.009 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:57.009 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e69d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:22:57.009 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1435 (0x7f431bfebd90) [ CSSD]2013-05-23 16:22:57.009 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebd90 (RPC#1435) tag(59b002c) sent to node 1 [ CSSD]2013-05-23 16:22:57.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(059b002c) incarnation(174631051) [ CSSD]2013-05-23 16:22:57.010 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:22:57.010 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebd90 (RPC#1435) state 6, flags 0x100 [ CSSD]2013-05-23 16:22:57.010 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfebd90, ret 0, client 0x7f431c10a560 member 0x7f431c0e69d0 [ CSSD]2013-05-23 16:22:57.010 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1435 [ CSSD]2013-05-23 16:22:57.010 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e69d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:22:57.212 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 16:22:57.212 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c11ab50) skgpid ospid 12184 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:22:58.572 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:22:58.573 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:22:58.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:22:58.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:22:58.763 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:00.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:00.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:00.770 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:01.229 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:23:01.229 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1436 (0x7f431bfebe10) [ CSSD]2013-05-23 16:23:01.229 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebe10 (RPC#1436) tag(59c002c) sent to node 1 [ CSSD]2013-05-23 16:23:01.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:01.230 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1436 (0x7f431bfebe10) [ CSSD]2013-05-23 16:23:01.230 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebe10 (RPC#1436) state 6, flags 0x202 [ CSSD]2013-05-23 16:23:01.230 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfebe10, ret 0, client (nil) [ CSSD]2013-05-23 16:23:01.230 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1436 [ CSSD]2013-05-23 16:23:02.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:02.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:02.780 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:03.576 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:03.576 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:04.786 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:04.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:04.807 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:06.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:06.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:06.813 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:07.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:07.415 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014534, gin 1, member 0 [ CSSD]2013-05-23 16:23:07.415 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:23:07.415 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:23:07.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:07.421 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:23:07.421 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:23:08.580 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:08.580 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:08.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:08.840 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:08.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:10.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:10.849 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:10.890 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:11.255 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:23:11.255 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1437 (0x7f431bfebe90) [ CSSD]2013-05-23 16:23:11.256 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebe90 (RPC#1437) tag(59d002c) sent to node 1 [ CSSD]2013-05-23 16:23:11.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:11.256 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1437 (0x7f431bfebe90) [ CSSD]2013-05-23 16:23:11.256 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebe90 (RPC#1437) state 6, flags 0x202 [ CSSD]2013-05-23 16:23:11.256 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfebe90, ret 0, client (nil) [ CSSD]2013-05-23 16:23:11.256 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1437 [ CSSD]2013-05-23 16:23:12.258 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 16:23:12.259 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:23:12.259 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 16:23:12.259 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d40) proc(0x7f431c048500) pid(12349) version 11:1:1:4 [ CSSD]2013-05-23 16:23:12.275 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c048500), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:23:12.275 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:23:12.275 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:23:12.275 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:12.275 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1438 (0x7f431bfebf10) [ CSSD]2013-05-23 16:23:12.275 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebf10 (RPC#1438) tag(59e002c) sent to node 1 [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(059e002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014535, gin 1, member 0 [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebf10 (RPC#1438) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfebf10, ret 0, client 0x7f431c10a560 member 0x7f431c0e69d0 [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:23:12.276 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1438 [ CSSD]2013-05-23 16:23:12.282 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:23:12.282 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:12.282 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e69d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:12.282 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1439 (0x7f431bfebf90) [ CSSD]2013-05-23 16:23:12.282 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfebf90 (RPC#1439) tag(59f002c) sent to node 1 [ CSSD]2013-05-23 16:23:12.283 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(059f002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:12.283 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:12.283 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfebf90 (RPC#1439) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:12.283 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfebf90, ret 0, client 0x7f431c10a560 member 0x7f431c0e69d0 [ CSSD]2013-05-23 16:23:12.283 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1439 [ CSSD]2013-05-23 16:23:12.283 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e69d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:23:12.485 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 16:23:12.485 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c137d40) skgpid ospid 12349 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:23:12.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:12.855 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:12.901 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:13.583 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:13.583 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:14.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:14.908 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:14.908 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:16.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:16.076 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014536, gin 1, member 0 [ CSSD]2013-05-23 16:23:16.076 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:23:16.076 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:23:16.115 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:16.115 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:23:16.115 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:23:16.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:16.913 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:16.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:18.586 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:18.586 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:18.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:18.925 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:18.964 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:19.888 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 16:23:19.889 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:23:19.890 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 16:23:19.890 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d40) proc(0x7f431c12f520) pid(12438) version 11:1:1:4 [ CSSD]2013-05-23 16:23:19.896 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12f520), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:23:19.896 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:23:19.896 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:23:19.896 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:19.896 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1440 (0x7f431bfec010) [ CSSD]2013-05-23 16:23:19.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec010 (RPC#1440) tag(5a0002c) sent to node 1 [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05a0002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014537, gin 1, member 0 [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec010 (RPC#1440) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec010, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:23:19.897 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1440 [ CSSD]2013-05-23 16:23:19.903 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:23:19.903 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:19.903 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:19.903 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1441 (0x7f431bfec090) [ CSSD]2013-05-23 16:23:19.903 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec090 (RPC#1441) tag(5a1002c) sent to node 1 [ CSSD]2013-05-23 16:23:19.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05a1002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:19.904 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:19.904 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec090 (RPC#1441) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:19.904 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfec090, ret 0, client 0x7f431c10a560 member 0x7f431c13eae0 [ CSSD]2013-05-23 16:23:19.904 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1441 [ CSSD]2013-05-23 16:23:19.904 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:23:20.106 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 16:23:20.106 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c137d40) skgpid ospid 12438 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:23:20.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:20.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:20.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:21.279 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:23:21.279 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1442 (0x7f431bfec110) [ CSSD]2013-05-23 16:23:21.279 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec110 (RPC#1442) tag(5a2002c) sent to node 1 [ CSSD]2013-05-23 16:23:21.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:21.280 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1442 (0x7f431bfec110) [ CSSD]2013-05-23 16:23:21.280 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec110 (RPC#1442) state 6, flags 0x202 [ CSSD]2013-05-23 16:23:21.280 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec110, ret 0, client (nil) [ CSSD]2013-05-23 16:23:21.280 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1442 [ CSSD]2013-05-23 16:23:22.589 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:22.589 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:23:22.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:22.667 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014538, gin 1, member 0 [ CSSD]2013-05-23 16:23:22.667 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:23:22.667 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:23:22.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:22.673 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:23:22.673 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:23:22.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:22.956 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:23.061 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:25.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:25.008 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:25.089 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:27.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:27.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:27.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:27.524 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 16:23:27.525 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:23:27.525 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 16:23:27.525 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c12f520) pid(12499) version 11:1:1:4 [ CSSD]2013-05-23 16:23:27.531 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12f520), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:23:27.531 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:23:27.531 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c10a560 with con 0x7f431c137d40, requested num -1 [ CSSD]2013-05-23 16:23:27.531 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:27.531 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1443 (0x7f431bfec190) [ CSSD]2013-05-23 16:23:27.531 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec190 (RPC#1443) tag(5a3002c) sent to node 1 [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05a3002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014539, gin 1, member 0 [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec190 (RPC#1443) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec190, ret 0, client 0x7f431c10a560 member 0x7f431c129710 [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:23:27.532 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1443 [ CSSD]2013-05-23 16:23:27.538 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:23:27.538 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:27.538 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:27.538 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1444 (0x7f431bfec210) [ CSSD]2013-05-23 16:23:27.538 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec210 (RPC#1444) tag(5a4002c) sent to node 1 [ CSSD]2013-05-23 16:23:27.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05a4002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:27.539 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:27.539 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec210 (RPC#1444) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:27.539 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfec210, ret 0, client 0x7f431c10a560 member 0x7f431c129710 [ CSSD]2013-05-23 16:23:27.539 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1444 [ CSSD]2013-05-23 16:23:27.539 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:23:27.592 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:27.593 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:27.741 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 16:23:27.741 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c0e6e20) skgpid ospid 12499 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:23:29.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:29.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:29.104 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:31.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:31.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:31.111 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:31.303 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:23:31.303 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1445 (0x7f431bfec290) [ CSSD]2013-05-23 16:23:31.303 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec290 (RPC#1445) tag(5a5002c) sent to node 1 [ CSSD]2013-05-23 16:23:31.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:31.304 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1445 (0x7f431bfec290) [ CSSD]2013-05-23 16:23:31.304 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec290 (RPC#1445) state 6, flags 0x202 [ CSSD]2013-05-23 16:23:31.304 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec290, ret 0, client (nil) [ CSSD]2013-05-23 16:23:31.304 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1445 [ CSSD]2013-05-23 16:23:32.596 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:32.596 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:33.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:33.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:33.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:35.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:35.109 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:35.158 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:37.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:37.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:37.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:37.599 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:37.600 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:37.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:37.916 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014540, gin 1, member 0 [ CSSD]2013-05-23 16:23:37.916 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:23:37.916 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:23:37.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:37.955 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:23:37.955 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:23:38.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:38.367 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014541, gin 1, member 0 [ CSSD]2013-05-23 16:23:38.367 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:23:38.367 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:23:38.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:38.374 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:23:38.374 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:23:38.509 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:23:38.510 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:23:38.511 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:23:38.512 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:23:38.514 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:23:38.514 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:23:39.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:39.146 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:39.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:41.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:41.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:41.181 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:41.326 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:23:41.327 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1446 (0x7f431bfec310) [ CSSD]2013-05-23 16:23:41.327 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec310 (RPC#1446) tag(5a6002c) sent to node 1 [ CSSD]2013-05-23 16:23:41.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:41.327 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1446 (0x7f431bfec310) [ CSSD]2013-05-23 16:23:41.327 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec310 (RPC#1446) state 6, flags 0x202 [ CSSD]2013-05-23 16:23:41.327 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec310, ret 0, client (nil) [ CSSD]2013-05-23 16:23:41.327 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1446 [ CSSD]2013-05-23 16:23:41.602 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:41.603 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:23:42.775 [1191196992] >TRACE: clssgmAllocProc: (0x22f1730) allocated [ CSSD]2013-05-23 16:23:42.776 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:23:42.776 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f1730 - 1,2,3 [ CSSD]2013-05-23 16:23:42.776 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22f1730) pid(12669) version 11:1:1:4 [ CSSD]2013-05-23 16:23:42.782 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f1730), client(1/0x2312950) [ CSSD]2013-05-23 16:23:42.782 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:23:42.782 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 16:23:42.782 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:42.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1447 (0x7f431bfec390) [ CSSD]2013-05-23 16:23:42.782 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec390 (RPC#1447) tag(5a7002c) sent to node 1 [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05a7002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014542, gin 1, member 0 [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec390 (RPC#1447) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec390, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 16:23:42.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1447 [ CSSD]2013-05-23 16:23:42.790 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:23:42.790 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:42.790 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:42.790 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1448 (0x7f431bfec410) [ CSSD]2013-05-23 16:23:42.790 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec410 (RPC#1448) tag(5a8002c) sent to node 1 [ CSSD]2013-05-23 16:23:42.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05a8002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:42.791 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:42.791 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec410 (RPC#1448) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:42.791 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfec410, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:23:42.791 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1448 [ CSSD]2013-05-23 16:23:42.791 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:23:42.993 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f1730 [ CSSD]2013-05-23 16:23:42.993 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f1730) con(0x205ecb0) skgpid ospid 12669 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:23:43.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:43.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:43.192 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:45.198 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:45.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:45.209 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:46.606 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:46.607 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:47.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:47.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:47.223 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:49.226 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:49.226 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:49.276 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:51.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:51.250 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:51.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:51.350 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:23:51.350 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1449 (0x7f431bfec490) [ CSSD]2013-05-23 16:23:51.350 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec490 (RPC#1449) tag(5a9002c) sent to node 1 [ CSSD]2013-05-23 16:23:51.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:51.351 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1449 (0x7f431bfec490) [ CSSD]2013-05-23 16:23:51.351 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec490 (RPC#1449) state 6, flags 0x202 [ CSSD]2013-05-23 16:23:51.351 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec490, ret 0, client (nil) [ CSSD]2013-05-23 16:23:51.351 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1449 [ CSSD]2013-05-23 16:23:51.610 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:51.611 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:53.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:53.166 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014543, gin 1, member 0 [ CSSD]2013-05-23 16:23:53.166 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:23:53.166 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:23:53.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:53.173 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:23:53.173 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:23:53.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:53.256 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:53.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:53.776 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0e5910) [ CSSD]2013-05-23 16:23:53.776 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0e5910), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:23:53.776 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0e5910), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:23:53.776 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 16:23:53.787 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:23:53.787 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:23:53.787 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:23:53.787 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:23:53.788 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b290 [ CSSD]2013-05-23 16:23:53.788 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b290) con(0x7f431c12f0c0) skgpid 7089 ospid 7089 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:23:55.310 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:55.310 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:55.319 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:56.614 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:23:56.614 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:23:56.821 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:23:56.822 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:23:56.822 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:23:56.822 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22d7cd0) pid(12802/12802) version 11:1:1:4 [ CSSD]2013-05-23 16:23:56.838 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7cd0), client(1/0x22f1730) [ CSSD]2013-05-23 16:23:56.838 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1730) [ CSSD]2013-05-23 16:23:56.838 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:23:56.838 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:23:57.056 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7cd0), client(2/0x2303710) [ CSSD]2013-05-23 16:23:57.056 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2303710) [ CSSD]2013-05-23 16:23:57.056 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:23:57.056 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 16:23:57.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:57.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:57.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:23:58.029 [1191196992] >TRACE: clssgmAllocProc: (0x22f8730) allocated [ CSSD]2013-05-23 16:23:58.031 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:23:58.031 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8730 - 1,2,3 [ CSSD]2013-05-23 16:23:58.031 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22fc5a0) proc(0x22f8730) pid(12829) version 11:1:1:4 [ CSSD]2013-05-23 16:23:58.038 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f8730), client(1/0x22f8860) [ CSSD]2013-05-23 16:23:58.038 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f8860) [ CSSD]2013-05-23 16:23:58.038 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f8860 with con 0x22dfd50, requested num -1 [ CSSD]2013-05-23 16:23:58.038 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:58.038 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1450 (0x7f431bfec510) [ CSSD]2013-05-23 16:23:58.038 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec510 (RPC#1450) tag(5aa002c) sent to node 1 [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05aa002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014544, gin 1, member 0 [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec510 (RPC#1450) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec510, ret 0, client 0x22f8860 member 0x22eed40 [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f8860 [ CSSD]2013-05-23 16:23:58.039 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1450 [ CSSD]2013-05-23 16:23:58.045 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f8860) [ CSSD]2013-05-23 16:23:58.045 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f8860), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:58.045 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eed40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:23:58.045 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1451 (0x7f431bfec590) [ CSSD]2013-05-23 16:23:58.045 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec590 (RPC#1451) tag(5ab002c) sent to node 1 [ CSSD]2013-05-23 16:23:58.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05ab002c) incarnation(174631051) [ CSSD]2013-05-23 16:23:58.045 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:23:58.045 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec590 (RPC#1451) state 6, flags 0x100 [ CSSD]2013-05-23 16:23:58.045 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfec590, ret 0, client 0x22f8860 member 0x22eed40 [ CSSD]2013-05-23 16:23:58.045 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1451 [ CSSD]2013-05-23 16:23:58.045 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eed40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:23:58.248 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f8730 [ CSSD]2013-05-23 16:23:58.248 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f8730) con(0x22fc5a0) skgpid ospid 12829 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:23:59.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:23:59.366 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:23:59.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:01.379 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:24:01.380 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1452 (0x7f431bfec610) [ CSSD]2013-05-23 16:24:01.380 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec610 (RPC#1452) tag(5ac002c) sent to node 1 [ CSSD]2013-05-23 16:24:01.380 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:01.380 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1452 (0x7f431bfec610) [ CSSD]2013-05-23 16:24:01.380 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec610 (RPC#1452) state 6, flags 0x202 [ CSSD]2013-05-23 16:24:01.380 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec610, ret 0, client (nil) [ CSSD]2013-05-23 16:24:01.381 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1452 [ CSSD]2013-05-23 16:24:01.402 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:01.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:01.402 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:01.618 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:01.618 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:02.054 [1191196992] >TRACE: clssgmAllocProc: (0x22eec70) allocated [ CSSD]2013-05-23 16:24:02.055 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:24:02.055 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22eec70 - 1,2,3 [ CSSD]2013-05-23 16:24:02.055 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22eec70) pid(12105/12105) version 11:1:1:4 [ CSSD]2013-05-23 16:24:02.072 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22eec70), client(1/0x22f7ce0) [ CSSD]2013-05-23 16:24:02.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:24:02.072 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:24:02.072 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:24:03.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:03.397 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:03.414 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:05.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:05.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:05.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:06.622 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:06.622 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:07.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:07.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:07.471 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:08.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:08.416 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014545, gin 1, member 0 [ CSSD]2013-05-23 16:24:08.416 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:24:08.416 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:24:08.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:08.422 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:24:08.422 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:24:09.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:09.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:09.477 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:11.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:24:11.404 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1453 (0x7f431bfec690) [ CSSD]2013-05-23 16:24:11.404 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec690 (RPC#1453) tag(5ad002c) sent to node 1 [ CSSD]2013-05-23 16:24:11.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:11.404 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1453 (0x7f431bfec690) [ CSSD]2013-05-23 16:24:11.404 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec690 (RPC#1453) state 6, flags 0x202 [ CSSD]2013-05-23 16:24:11.404 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec690, ret 0, client (nil) [ CSSD]2013-05-23 16:24:11.405 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1453 [ CSSD]2013-05-23 16:24:11.433 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:11.433 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:11.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:11.625 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:11.626 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:13.285 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:24:13.287 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:24:13.287 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:24:13.287 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(12979) version 11:1:1:4 [ CSSD]2013-05-23 16:24:13.294 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231ed60), client(1/0x2302940) [ CSSD]2013-05-23 16:24:13.294 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:13.294 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302940 with con 0x2302640, requested num -1 [ CSSD]2013-05-23 16:24:13.294 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:13.294 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1454 (0x7f431bfec710) [ CSSD]2013-05-23 16:24:13.294 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec710 (RPC#1454) tag(5ae002c) sent to node 1 [ CSSD]2013-05-23 16:24:13.294 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05ae002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:13.294 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014546, gin 1, member 0 [ CSSD]2013-05-23 16:24:13.294 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:24:13.295 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:24:13.295 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:24:13.295 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec710 (RPC#1454) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:13.295 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec710, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:13.295 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302940 [ CSSD]2013-05-23 16:24:13.295 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1454 [ CSSD]2013-05-23 16:24:13.302 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:13.302 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:13.302 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:13.302 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1455 (0x7f431bfec790) [ CSSD]2013-05-23 16:24:13.302 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec790 (RPC#1455) tag(5af002c) sent to node 1 [ CSSD]2013-05-23 16:24:13.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05af002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:13.302 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:13.302 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec790 (RPC#1455) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:13.302 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfec790, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:13.303 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1455 [ CSSD]2013-05-23 16:24:13.303 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:24:13.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:13.457 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:13.505 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:24:13.505 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 12979 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:24:13.595 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:15.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:15.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:15.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:15.628 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:15.629 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:24:16.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:16.335 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014547, gin 1, member 0 [ CSSD]2013-05-23 16:24:16.335 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:24:16.335 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:24:16.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:16.341 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:24:16.341 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:24:17.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:17.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:17.672 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:19.518 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:19.518 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:19.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:20.147 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:24:20.147 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:24:20.148 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:24:20.148 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(13065) version 11:1:1:4 [ CSSD]2013-05-23 16:24:20.155 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231ed60), client(1/0x2302940) [ CSSD]2013-05-23 16:24:20.155 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:20.155 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302940 with con 0x2302640, requested num -1 [ CSSD]2013-05-23 16:24:20.155 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:20.155 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1456 (0x7f431bfec810) [ CSSD]2013-05-23 16:24:20.155 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec810 (RPC#1456) tag(5b0002c) sent to node 1 [ CSSD]2013-05-23 16:24:20.155 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05b0002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:20.155 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014548, gin 1, member 0 [ CSSD]2013-05-23 16:24:20.155 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:24:20.156 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:24:20.156 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:24:20.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec810 (RPC#1456) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:20.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec810, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:20.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302940 [ CSSD]2013-05-23 16:24:20.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1456 [ CSSD]2013-05-23 16:24:20.161 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:20.161 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:20.161 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:20.161 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1457 (0x7f431bfec890) [ CSSD]2013-05-23 16:24:20.161 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec890 (RPC#1457) tag(5b1002c) sent to node 1 [ CSSD]2013-05-23 16:24:20.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05b1002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:20.162 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:20.162 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec890 (RPC#1457) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:20.162 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfec890, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:20.162 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1457 [ CSSD]2013-05-23 16:24:20.162 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:24:20.364 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:24:20.364 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 13065 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:24:20.632 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:20.632 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:21.426 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:24:21.427 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1458 (0x7f431bfec910) [ CSSD]2013-05-23 16:24:21.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec910 (RPC#1458) tag(5b2002c) sent to node 1 [ CSSD]2013-05-23 16:24:21.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:21.428 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1458 (0x7f431bfec910) [ CSSD]2013-05-23 16:24:21.428 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec910 (RPC#1458) state 6, flags 0x202 [ CSSD]2013-05-23 16:24:21.428 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfec910, ret 0, client (nil) [ CSSD]2013-05-23 16:24:21.428 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1458 [ CSSD]2013-05-23 16:24:21.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:21.543 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:21.682 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:23.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:23.550 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:23.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:23.674 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014549, gin 1, member 0 [ CSSD]2013-05-23 16:24:23.674 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:24:23.674 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:24:23.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:23.680 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:24:23.680 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:24:23.774 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:25.395 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:24:25.395 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:24:25.396 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:24:25.396 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:24:25.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:25.479 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014550, gin 1, member 0 [ CSSD]2013-05-23 16:24:25.479 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x205e170/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:24:25.479 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:24:25.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:25.485 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:24:25.485 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:24:25.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:25.560 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:25.634 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:25.634 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:25.780 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:27.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:27.566 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:27.873 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:28.546 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:24:28.547 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:24:28.548 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:24:28.548 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(13128) version 11:1:1:4 [ CSSD]2013-05-23 16:24:28.565 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231ed60), client(1/0x2302940) [ CSSD]2013-05-23 16:24:28.565 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:28.565 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302940 with con 0x2302640, requested num -1 [ CSSD]2013-05-23 16:24:28.565 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:28.565 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1459 (0x7f431bfec990) [ CSSD]2013-05-23 16:24:28.565 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfec990 (RPC#1459) tag(5b3002c) sent to node 1 [ CSSD]2013-05-23 16:24:28.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05b3002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:28.565 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014551, gin 1, member 0 [ CSSD]2013-05-23 16:24:28.565 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:24:28.566 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:24:28.566 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:24:28.566 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfec990 (RPC#1459) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:28.566 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfec990, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:28.566 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302940 [ CSSD]2013-05-23 16:24:28.566 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1459 [ CSSD]2013-05-23 16:24:28.571 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:28.571 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:28.571 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:28.571 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1460 (0x7f431bfeca10) [ CSSD]2013-05-23 16:24:28.572 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeca10 (RPC#1460) tag(5b4002c) sent to node 1 [ CSSD]2013-05-23 16:24:28.572 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05b4002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:28.572 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:28.572 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeca10 (RPC#1460) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:28.572 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeca10, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:28.572 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1460 [ CSSD]2013-05-23 16:24:28.572 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:24:28.775 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:24:28.775 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 13128 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:24:29.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:29.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:29.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:30.637 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:30.637 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:31.451 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:24:31.451 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1461 (0x7f431bfeca90) [ CSSD]2013-05-23 16:24:31.451 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeca90 (RPC#1461) tag(5b5002c) sent to node 1 [ CSSD]2013-05-23 16:24:31.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:31.452 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1461 (0x7f431bfeca90) [ CSSD]2013-05-23 16:24:31.452 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeca90 (RPC#1461) state 6, flags 0x202 [ CSSD]2013-05-23 16:24:31.452 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeca90, ret 0, client (nil) [ CSSD]2013-05-23 16:24:31.452 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1461 [ CSSD]2013-05-23 16:24:31.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:31.625 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:31.884 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:33.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:33.651 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:33.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:35.639 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:35.639 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:35.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:35.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:35.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:37.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:37.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:37.913 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:38.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:38.926 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014552, gin 1, member 0 [ CSSD]2013-05-23 16:24:38.926 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:24:38.926 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:24:38.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:38.933 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:24:38.933 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:24:39.642 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:39.642 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:24:39.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:39.719 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:39.919 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:41.475 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:24:41.475 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1462 (0x7f431bfecb10) [ CSSD]2013-05-23 16:24:41.475 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecb10 (RPC#1462) tag(5b6002c) sent to node 1 [ CSSD]2013-05-23 16:24:41.476 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:41.476 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1462 (0x7f431bfecb10) [ CSSD]2013-05-23 16:24:41.476 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecb10 (RPC#1462) state 6, flags 0x202 [ CSSD]2013-05-23 16:24:41.476 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfecb10, ret 0, client (nil) [ CSSD]2013-05-23 16:24:41.476 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1462 [ CSSD]2013-05-23 16:24:41.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:41.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:41.924 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:43.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:43.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:43.805 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 16:24:43.806 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:24:43.807 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 16:24:43.807 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302cd0) proc(0x22d7350) pid(13270) version 11:1:1:4 [ CSSD]2013-05-23 16:24:43.812 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d7350), client(1/0x231ee10) [ CSSD]2013-05-23 16:24:43.812 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x231ee10) [ CSSD]2013-05-23 16:24:43.813 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x231ee10 with con 0x22d7590, requested num -1 [ CSSD]2013-05-23 16:24:43.813 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:43.813 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1463 (0x7f431bfecb90) [ CSSD]2013-05-23 16:24:43.813 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecb90 (RPC#1463) tag(5b7002c) sent to node 1 [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05b7002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014553, gin 1, member 0 [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f19b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecb90 (RPC#1463) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfecb90, ret 0, client 0x231ee10 member 0x231ec90 [ CSSD]2013-05-23 16:24:43.813 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x231ee10 [ CSSD]2013-05-23 16:24:43.814 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1463 [ CSSD]2013-05-23 16:24:43.819 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x231ee10) [ CSSD]2013-05-23 16:24:43.819 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x231ee10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:43.819 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231ec90) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:43.819 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1464 (0x7f431bfecc10) [ CSSD]2013-05-23 16:24:43.819 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecc10 (RPC#1464) tag(5b8002c) sent to node 1 [ CSSD]2013-05-23 16:24:43.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05b8002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:43.820 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:43.820 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecc10 (RPC#1464) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:43.820 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfecc10, ret 0, client 0x231ee10 member 0x231ec90 [ CSSD]2013-05-23 16:24:43.820 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1464 [ CSSD]2013-05-23 16:24:43.820 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231ec90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:24:43.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:44.023 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 16:24:44.023 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x2302cd0) skgpid ospid 13270 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:24:44.644 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:44.645 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:45.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:45.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:45.981 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:47.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:47.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:47.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:49.647 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:49.648 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:49.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:49.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:49.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:51.500 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:24:51.501 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1465 (0x7f431bfecc90) [ CSSD]2013-05-23 16:24:51.501 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecc90 (RPC#1465) tag(5b9002c) sent to node 1 [ CSSD]2013-05-23 16:24:51.501 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:51.501 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1465 (0x7f431bfecc90) [ CSSD]2013-05-23 16:24:51.501 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecc90 (RPC#1465) state 6, flags 0x202 [ CSSD]2013-05-23 16:24:51.501 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfecc90, ret 0, client (nil) [ CSSD]2013-05-23 16:24:51.502 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1465 [ CSSD]2013-05-23 16:24:51.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:51.826 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:51.985 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:53.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:53.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:53.985 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:54.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:54.177 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014554, gin 1, member 0 [ CSSD]2013-05-23 16:24:54.177 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:24:54.177 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:24:54.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:54.185 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:24:54.185 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:24:54.650 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:54.650 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:55.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:55.842 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:55.990 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:57.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:57.848 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:57.983 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:24:59.055 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:24:59.056 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:24:59.057 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:24:59.057 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(13441) version 11:1:1:4 [ CSSD]2013-05-23 16:24:59.062 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231ed60), client(1/0x2302940) [ CSSD]2013-05-23 16:24:59.062 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:59.062 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302940 with con 0x2302640, requested num -1 [ CSSD]2013-05-23 16:24:59.062 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:59.062 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1466 (0x7f431bfecd10) [ CSSD]2013-05-23 16:24:59.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecd10 (RPC#1466) tag(5ba002c) sent to node 1 [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05ba002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014555, gin 1, member 0 [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecd10 (RPC#1466) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfecd10, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302940 [ CSSD]2013-05-23 16:24:59.063 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1466 [ CSSD]2013-05-23 16:24:59.069 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:24:59.069 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:59.069 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:24:59.069 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1467 (0x7f431bfecd90) [ CSSD]2013-05-23 16:24:59.070 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecd90 (RPC#1467) tag(5bb002c) sent to node 1 [ CSSD]2013-05-23 16:24:59.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05bb002c) incarnation(174631051) [ CSSD]2013-05-23 16:24:59.070 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:24:59.070 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecd90 (RPC#1467) state 6, flags 0x100 [ CSSD]2013-05-23 16:24:59.070 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfecd90, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:24:59.070 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1467 [ CSSD]2013-05-23 16:24:59.070 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:24:59.273 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:24:59.273 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 13441 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:24:59.653 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:24:59.653 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:24:59.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:24:59.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:24:59.989 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:01.530 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:25:01.531 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1468 (0x7f431bfece10) [ CSSD]2013-05-23 16:25:01.531 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfece10 (RPC#1468) tag(5bc002c) sent to node 1 [ CSSD]2013-05-23 16:25:01.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:01.531 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1468 (0x7f431bfece10) [ CSSD]2013-05-23 16:25:01.531 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfece10 (RPC#1468) state 6, flags 0x202 [ CSSD]2013-05-23 16:25:01.531 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfece10, ret 0, client (nil) [ CSSD]2013-05-23 16:25:01.531 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1468 [ CSSD]2013-05-23 16:25:01.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:01.865 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:01.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:03.656 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:03.656 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:25:03.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:03.918 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:04.086 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:05.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:05.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:06.091 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:07.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:07.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:08.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:08.363 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:25:08.364 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:25:08.364 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:25:08.364 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(13579) version 11:1:1:4 [ CSSD]2013-05-23 16:25:08.659 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:08.659 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:08.981 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:25:08.981 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 13579 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:25:09.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:09.438 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014556, gin 1, member 0 [ CSSD]2013-05-23 16:25:09.438 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:25:09.438 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:25:09.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:09.444 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:25:09.444 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:25:10.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:10.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:10.110 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:11.557 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:25:11.558 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1469 (0x7f431bfece90) [ CSSD]2013-05-23 16:25:11.558 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfece90 (RPC#1469) tag(5bd002c) sent to node 1 [ CSSD]2013-05-23 16:25:11.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:11.558 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1469 (0x7f431bfece90) [ CSSD]2013-05-23 16:25:11.558 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfece90 (RPC#1469) state 6, flags 0x202 [ CSSD]2013-05-23 16:25:11.558 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfece90, ret 0, client (nil) [ CSSD]2013-05-23 16:25:11.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1469 [ CSSD]2013-05-23 16:25:12.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:12.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:12.116 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:13.662 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:13.662 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:14.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:14.038 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:14.125 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:14.304 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:25:14.304 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:25:14.305 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:25:14.305 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(13721) version 11:1:1:4 [ CSSD]2013-05-23 16:25:14.311 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231ed60), client(1/0x2302940) [ CSSD]2013-05-23 16:25:14.311 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:25:14.311 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302940 with con 0x2302640, requested num -1 [ CSSD]2013-05-23 16:25:14.311 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:14.311 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1470 (0x7f431bfecf10) [ CSSD]2013-05-23 16:25:14.311 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecf10 (RPC#1470) tag(5be002c) sent to node 1 [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05be002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014557, gin 1, member 0 [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecf10 (RPC#1470) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfecf10, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302940 [ CSSD]2013-05-23 16:25:14.312 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1470 [ CSSD]2013-05-23 16:25:14.318 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:25:14.318 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:14.318 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:14.318 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1471 (0x7f431bfecf90) [ CSSD]2013-05-23 16:25:14.318 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfecf90 (RPC#1471) tag(5bf002c) sent to node 1 [ CSSD]2013-05-23 16:25:14.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05bf002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:14.318 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:14.318 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfecf90 (RPC#1471) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:14.318 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfecf90, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:25:14.319 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1471 [ CSSD]2013-05-23 16:25:14.319 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:25:14.521 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:25:14.521 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 13721 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:25:16.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:16.044 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:16.130 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:16.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:16.598 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014558, gin 1, member 0 [ CSSD]2013-05-23 16:25:16.598 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:25:16.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:25:16.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:16.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:25:16.605 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dfe20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:25:18.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:18.069 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:18.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:18.665 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:18.665 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:20.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:20.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:20.189 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:20.412 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 16:25:20.413 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:25:20.413 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 16:25:20.413 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x231ed60) pid(13807) version 11:1:1:4 [ CSSD]2013-05-23 16:25:20.419 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x231ed60), client(1/0x2302940) [ CSSD]2013-05-23 16:25:20.419 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:25:20.419 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302940 with con 0x2302640, requested num -1 [ CSSD]2013-05-23 16:25:20.419 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:20.419 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1472 (0x7f431bfed010) [ CSSD]2013-05-23 16:25:20.420 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed010 (RPC#1472) tag(5c0002c) sent to node 1 [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05c0002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014559, gin 1, member 0 [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23178a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed010 (RPC#1472) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed010, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:25:20.420 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302940 [ CSSD]2013-05-23 16:25:20.421 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1472 [ CSSD]2013-05-23 16:25:20.426 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302940) [ CSSD]2013-05-23 16:25:20.426 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:20.426 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:20.426 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1473 (0x7f431bfed090) [ CSSD]2013-05-23 16:25:20.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed090 (RPC#1473) tag(5c1002c) sent to node 1 [ CSSD]2013-05-23 16:25:20.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05c1002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:20.427 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:20.427 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed090 (RPC#1473) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:20.427 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed090, ret 0, client 0x2302940 member 0x22d7420 [ CSSD]2013-05-23 16:25:20.427 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1473 [ CSSD]2013-05-23 16:25:20.427 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:25:20.630 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 16:25:20.630 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x2302d80) skgpid ospid 13807 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:25:21.580 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:25:21.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1474 (0x7f431bfed110) [ CSSD]2013-05-23 16:25:21.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed110 (RPC#1474) tag(5c2002c) sent to node 1 [ CSSD]2013-05-23 16:25:21.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:21.581 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1474 (0x7f431bfed110) [ CSSD]2013-05-23 16:25:21.581 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed110 (RPC#1474) state 6, flags 0x202 [ CSSD]2013-05-23 16:25:21.581 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfed110, ret 0, client (nil) [ CSSD]2013-05-23 16:25:21.582 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1474 [ CSSD]2013-05-23 16:25:22.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:22.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:22.195 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:22.667 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:22.667 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:25:23.527 [1191196992] >TRACE: clssgmAllocProc: (0x22d7550) allocated [ CSSD]2013-05-23 16:25:23.529 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:25:23.529 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7550 - 1,2,3 [ CSSD]2013-05-23 16:25:23.529 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d80) proc(0x22d7550) pid(13799/13799) version 11:1:1:4 [ CSSD]2013-05-23 16:25:23.535 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d7550), client(1/0x23029c0) [ CSSD]2013-05-23 16:25:23.535 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:25:23.535 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:25:23.535 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:25:24.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:24.138 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:24.287 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:24.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:24.687 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014560, gin 1, member 0 [ CSSD]2013-05-23 16:25:24.687 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f19b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:25:24.687 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:25:24.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:24.693 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:25:24.693 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23027c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:25:26.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:26.143 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:26.293 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:27.670 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:27.670 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:28.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:28.167 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:28.305 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:29.553 [1191196992] >TRACE: clssgmAllocProc: (0x22f2ed0) allocated [ CSSD]2013-05-23 16:25:29.554 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:25:29.554 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2ed0 - 1,2,3 [ CSSD]2013-05-23 16:25:29.554 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317960) proc(0x22f2ed0) pid(13873) version 11:1:1:4 [ CSSD]2013-05-23 16:25:29.560 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f2ed0), client(1/0x22f4d90) [ CSSD]2013-05-23 16:25:29.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4d90) [ CSSD]2013-05-23 16:25:29.560 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4d90 with con 0x22f30d0, requested num -1 [ CSSD]2013-05-23 16:25:29.560 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:29.560 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1475 (0x7f431bfed190) [ CSSD]2013-05-23 16:25:29.560 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed190 (RPC#1475) tag(5c3002c) sent to node 1 [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05c3002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014561, gin 1, member 0 [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4f00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed190 (RPC#1475) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed190, ret 0, client 0x22f4d90 member 0x22f2cc0 [ CSSD]2013-05-23 16:25:29.560 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4d90 [ CSSD]2013-05-23 16:25:29.561 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1475 [ CSSD]2013-05-23 16:25:29.566 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4d90) [ CSSD]2013-05-23 16:25:29.566 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4d90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:29.566 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:29.566 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1476 (0x7f431bfed210) [ CSSD]2013-05-23 16:25:29.566 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed210 (RPC#1476) tag(5c4002c) sent to node 1 [ CSSD]2013-05-23 16:25:29.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05c4002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:29.567 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:29.567 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed210 (RPC#1476) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:29.567 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed210, ret 0, client 0x22f4d90 member 0x22f2cc0 [ CSSD]2013-05-23 16:25:29.567 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1476 [ CSSD]2013-05-23 16:25:29.567 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:25:29.770 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2ed0 [ CSSD]2013-05-23 16:25:29.770 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2ed0) con(0x2317960) skgpid ospid 13873 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:25:30.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:30.222 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:30.311 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:30.899 [1191196992] >TRACE: clssgmAllocProc: (0x22f2ed0) allocated [ CSSD]2013-05-23 16:25:30.900 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:25:30.900 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2ed0 - 1,2,3 [ CSSD]2013-05-23 16:25:30.901 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317960) proc(0x22f2ed0) pid(12800/12800) version 11:1:1:4 [ CSSD]2013-05-23 16:25:30.906 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f2ed0), client(1/0x22f4c00) [ CSSD]2013-05-23 16:25:30.906 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f4c00) [ CSSD]2013-05-23 16:25:30.906 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:25:30.906 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:25:31.392 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d7550), client(2/0x22f5100) [ CSSD]2013-05-23 16:25:31.392 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f5100) [ CSSD]2013-05-23 16:25:31.392 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:25:31.392 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:25:31.605 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:25:31.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1477 (0x7f431bfed290) [ CSSD]2013-05-23 16:25:31.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed290 (RPC#1477) tag(5c5002c) sent to node 1 [ CSSD]2013-05-23 16:25:31.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:31.606 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1477 (0x7f431bfed290) [ CSSD]2013-05-23 16:25:31.606 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed290 (RPC#1477) state 6, flags 0x202 [ CSSD]2013-05-23 16:25:31.606 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfed290, ret 0, client (nil) [ CSSD]2013-05-23 16:25:31.606 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1477 [ CSSD]2013-05-23 16:25:32.227 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:32.227 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:32.316 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:32.673 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:32.673 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:34.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:34.229 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:34.408 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:36.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:36.254 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:36.432 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:37.676 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:37.676 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:38.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:38.260 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:38.485 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:39.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:39.941 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014562, gin 1, member 0 [ CSSD]2013-05-23 16:25:39.941 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2317dd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:25:39.941 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:25:39.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:39.948 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:25:39.948 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4ef0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:25:40.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:40.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:40.491 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:41.628 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:25:41.628 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1478 (0x7f431bfed310) [ CSSD]2013-05-23 16:25:41.629 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed310 (RPC#1478) tag(5c6002c) sent to node 1 [ CSSD]2013-05-23 16:25:41.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:41.629 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1478 (0x7f431bfed310) [ CSSD]2013-05-23 16:25:41.629 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed310 (RPC#1478) state 6, flags 0x202 [ CSSD]2013-05-23 16:25:41.629 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfed310, ret 0, client (nil) [ CSSD]2013-05-23 16:25:41.629 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1478 [ CSSD]2013-05-23 16:25:42.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:42.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:42.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:42.679 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:42.679 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:44.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:44.330 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:44.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:44.803 [1191196992] >TRACE: clssgmAllocProc: (0x2319100) allocated [ CSSD]2013-05-23 16:25:44.804 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:25:44.805 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2319100 - 1,2,3 [ CSSD]2013-05-23 16:25:44.805 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2318270) proc(0x2319100) pid(14023) version 11:1:1:4 [ CSSD]2013-05-23 16:25:44.822 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2319100), client(1/0x22edcf0) [ CSSD]2013-05-23 16:25:44.822 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22edcf0) [ CSSD]2013-05-23 16:25:44.822 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22edcf0 with con 0x2318ef0, requested num -1 [ CSSD]2013-05-23 16:25:44.822 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:44.822 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1479 (0x7f431bfed390) [ CSSD]2013-05-23 16:25:44.822 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed390 (RPC#1479) tag(5c7002c) sent to node 1 [ CSSD]2013-05-23 16:25:44.822 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05c7002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:44.822 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014563, gin 1, member 0 [ CSSD]2013-05-23 16:25:44.822 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ede60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:25:44.823 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:25:44.823 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:25:44.823 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed390 (RPC#1479) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:44.823 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed390, ret 0, client 0x22edcf0 member 0x23193f0 [ CSSD]2013-05-23 16:25:44.823 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22edcf0 [ CSSD]2013-05-23 16:25:44.823 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1479 [ CSSD]2013-05-23 16:25:44.830 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22edcf0) [ CSSD]2013-05-23 16:25:44.830 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22edcf0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:44.830 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23193f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:25:44.830 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1480 (0x7f431bfed410) [ CSSD]2013-05-23 16:25:44.830 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed410 (RPC#1480) tag(5c8002c) sent to node 1 [ CSSD]2013-05-23 16:25:44.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05c8002c) incarnation(174631051) [ CSSD]2013-05-23 16:25:44.830 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:25:44.830 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed410 (RPC#1480) state 6, flags 0x100 [ CSSD]2013-05-23 16:25:44.830 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed410, ret 0, client 0x22edcf0 member 0x23193f0 [ CSSD]2013-05-23 16:25:44.830 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1480 [ CSSD]2013-05-23 16:25:44.830 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23193f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:25:45.033 [1191196992] >TRACE: clssgmDeadProc: proc 0x2319100 [ CSSD]2013-05-23 16:25:45.033 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2319100) con(0x2318270) skgpid ospid 14023 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:25:46.141 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f2ed0), client(2/0x2317dd0) [ CSSD]2013-05-23 16:25:46.141 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2317dd0) [ CSSD]2013-05-23 16:25:46.141 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:25:46.141 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:25:46.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:46.354 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:46.514 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:46.681 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:46.681 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:25:48.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:48.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:48.526 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:50.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:50.373 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:50.531 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:51.653 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:25:51.654 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1481 (0x7f431bfed490) [ CSSD]2013-05-23 16:25:51.654 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed490 (RPC#1481) tag(5c9002c) sent to node 1 [ CSSD]2013-05-23 16:25:51.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:51.654 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1481 (0x7f431bfed490) [ CSSD]2013-05-23 16:25:51.654 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed490 (RPC#1481) state 6, flags 0x202 [ CSSD]2013-05-23 16:25:51.654 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfed490, ret 0, client (nil) [ CSSD]2013-05-23 16:25:51.654 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1481 [ CSSD]2013-05-23 16:25:51.684 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:51.684 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:52.378 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:52.378 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:52.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:54.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:54.431 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:54.589 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:55.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:55.191 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014564, gin 1, member 0 [ CSSD]2013-05-23 16:25:55.191 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2317ff0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:25:55.191 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:25:55.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:55.197 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:25:55.197 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4fc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:25:56.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:56.456 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:56.595 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:25:56.687 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:25:56.687 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:25:58.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:25:58.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:25:58.605 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:00.065 [1191196992] >TRACE: clssgmAllocProc: (0x22ed960) allocated [ CSSD]2013-05-23 16:26:00.066 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:26:00.067 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed960 - 1,2,3 [ CSSD]2013-05-23 16:26:00.067 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2318560) proc(0x22ed960) pid(14173) version 11:1:1:4 [ CSSD]2013-05-23 16:26:00.072 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed960), client(1/0x22ee140) [ CSSD]2013-05-23 16:26:00.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee140) [ CSSD]2013-05-23 16:26:00.072 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee140 with con 0x22edc30, requested num -1 [ CSSD]2013-05-23 16:26:00.072 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:00.072 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1482 (0x7f431bfed510) [ CSSD]2013-05-23 16:26:00.073 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed510 (RPC#1482) tag(5ca002c) sent to node 1 [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05ca002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014565, gin 1, member 0 [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e33d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed510 (RPC#1482) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed510, ret 0, client 0x22ee140 member 0x23192d0 [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee140 [ CSSD]2013-05-23 16:26:00.073 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1482 [ CSSD]2013-05-23 16:26:00.079 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee140) [ CSSD]2013-05-23 16:26:00.079 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:00.079 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23192d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:00.079 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1483 (0x7f431bfed590) [ CSSD]2013-05-23 16:26:00.079 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed590 (RPC#1483) tag(5cb002c) sent to node 1 [ CSSD]2013-05-23 16:26:00.079 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05cb002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:00.079 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:00.079 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed590 (RPC#1483) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:00.079 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed590, ret 0, client 0x22ee140 member 0x23192d0 [ CSSD]2013-05-23 16:26:00.080 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1483 [ CSSD]2013-05-23 16:26:00.080 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23192d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:26:00.282 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed960 [ CSSD]2013-05-23 16:26:00.282 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed960) con(0x2318560) skgpid ospid 14173 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:00.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:00.473 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:00.610 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:01.675 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:26:01.675 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1484 (0x7f431bfed610) [ CSSD]2013-05-23 16:26:01.675 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed610 (RPC#1484) tag(5cc002c) sent to node 1 [ CSSD]2013-05-23 16:26:01.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:01.676 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1484 (0x7f431bfed610) [ CSSD]2013-05-23 16:26:01.676 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed610 (RPC#1484) state 6, flags 0x202 [ CSSD]2013-05-23 16:26:01.676 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfed610, ret 0, client (nil) [ CSSD]2013-05-23 16:26:01.676 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1484 [ CSSD]2013-05-23 16:26:01.690 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:01.690 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:02.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:02.479 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:02.604 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:04.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:04.531 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:04.610 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:06.537 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:06.537 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:06.615 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:06.692 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:06.693 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:08.562 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:08.562 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:08.627 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:10.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:10.456 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014566, gin 1, member 0 [ CSSD]2013-05-23 16:26:10.457 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2317ff0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:26:10.457 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:26:10.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:10.463 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:26:10.463 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4fc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:26:10.632 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:10.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:10.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:10.695 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:10.695 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:26:11.698 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:26:11.698 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1485 (0x7f431bfed690) [ CSSD]2013-05-23 16:26:11.698 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed690 (RPC#1485) tag(5cd002c) sent to node 1 [ CSSD]2013-05-23 16:26:11.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:11.699 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1485 (0x7f431bfed690) [ CSSD]2013-05-23 16:26:11.699 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed690 (RPC#1485) state 6, flags 0x202 [ CSSD]2013-05-23 16:26:11.699 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfed690, ret 0, client (nil) [ CSSD]2013-05-23 16:26:11.699 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1485 [ CSSD]2013-05-23 16:26:12.634 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:12.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:12.679 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:14.640 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:14.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:14.731 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:15.314 [1191196992] >TRACE: clssgmAllocProc: (0x22ed960) allocated [ CSSD]2013-05-23 16:26:15.315 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:26:15.315 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed960 - 1,2,3 [ CSSD]2013-05-23 16:26:15.315 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23184b0) proc(0x22ed960) pid(14339) version 11:1:1:4 [ CSSD]2013-05-23 16:26:15.332 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed960), client(1/0x22ee090) [ CSSD]2013-05-23 16:26:15.332 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee090) [ CSSD]2013-05-23 16:26:15.332 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee090 with con 0x23192b0, requested num -1 [ CSSD]2013-05-23 16:26:15.332 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:15.332 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1486 (0x7f431bfed710) [ CSSD]2013-05-23 16:26:15.332 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed710 (RPC#1486) tag(5ce002c) sent to node 1 [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05ce002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014567, gin 1, member 0 [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed710 (RPC#1486) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed710, ret 0, client 0x22ee090 member 0x22e3600 [ CSSD]2013-05-23 16:26:15.332 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee090 [ CSSD]2013-05-23 16:26:15.333 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1486 [ CSSD]2013-05-23 16:26:15.339 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee090) [ CSSD]2013-05-23 16:26:15.339 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee090), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:15.339 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e3600) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:15.339 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1487 (0x7f431bfed790) [ CSSD]2013-05-23 16:26:15.339 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed790 (RPC#1487) tag(5cf002c) sent to node 1 [ CSSD]2013-05-23 16:26:15.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05cf002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:15.339 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:15.339 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed790 (RPC#1487) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:15.339 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed790, ret 0, client 0x22ee090 member 0x22e3600 [ CSSD]2013-05-23 16:26:15.340 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1487 [ CSSD]2013-05-23 16:26:15.340 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e3600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:26:15.542 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed960 [ CSSD]2013-05-23 16:26:15.542 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed960) con(0x23184b0) skgpid ospid 14339 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:15.698 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:15.698 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:16.646 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:16.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:16.757 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:16.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:16.856 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014568, gin 1, member 0 [ CSSD]2013-05-23 16:26:16.856 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2317f40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:26:16.856 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:26:16.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:16.862 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:26:16.862 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4fc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:26:17.584 [1191196992] >TRACE: clssgmAllocProc: (0x22ed960) allocated [ CSSD]2013-05-23 16:26:17.585 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:26:17.585 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed960 - 1,2,3 [ CSSD]2013-05-23 16:26:17.585 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23184b0) proc(0x22ed960) pid(14396) version 11:1:1:4 [ CSSD]2013-05-23 16:26:17.602 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed960), client(1/0x22ee090) [ CSSD]2013-05-23 16:26:17.602 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee090) [ CSSD]2013-05-23 16:26:17.602 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22ee090 with con 0x23192b0, requested num -1 [ CSSD]2013-05-23 16:26:17.602 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:26:17.602 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1488 (0x7f431bfed810) [ CSSD]2013-05-23 16:26:17.602 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed810 (RPC#1488) tag(5d0002c) sent to node 1 [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05d0002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014569, gin 1, member 0 [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3300/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed810 (RPC#1488) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed810, ret 0, client 0x22ee090 member 0x22e3600 [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee090 [ CSSD]2013-05-23 16:26:17.603 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1488 [ CSSD]2013-05-23 16:26:17.610 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee090) [ CSSD]2013-05-23 16:26:17.610 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee090), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:26:17.610 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e3600) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:26:17.610 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1489 (0x7f431bfed890) [ CSSD]2013-05-23 16:26:17.610 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed890 (RPC#1489) tag(5d1002c) sent to node 1 [ CSSD]2013-05-23 16:26:17.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05d1002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:17.611 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:26:17.611 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed890 (RPC#1489) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:17.611 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed890, ret 0, client 0x22ee090 member 0x22e3600 [ CSSD]2013-05-23 16:26:17.611 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1489 [ CSSD]2013-05-23 16:26:17.611 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22e3600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:26:17.717 [1191196992] >TRACE: clssgmAllocProc: (0x2320f80) allocated [ CSSD]2013-05-23 16:26:17.718 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:26:17.719 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2320f80 - 1,2,3 [ CSSD]2013-05-23 16:26:17.719 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee1a0) proc(0x2320f80) pid(14407) version 11:1:1:4 [ CSSD]2013-05-23 16:26:17.745 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:26:17.746 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:26:17.749 [1191196992] >TRACE: clssgmDeadProc: proc 0x2320f80 [ CSSD]2013-05-23 16:26:17.749 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2320f80) con(0x22ee1a0) skgpid ospid 14407 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:17.813 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed960 [ CSSD]2013-05-23 16:26:17.813 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed960) con(0x23184b0) skgpid ospid 14396 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:18.697 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:18.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:18.763 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:20.664 [1191196992] >TRACE: clssgmAllocProc: (0x22ed960) allocated [ CSSD]2013-05-23 16:26:20.666 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:26:20.666 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed960 - 1,2,3 [ CSSD]2013-05-23 16:26:20.666 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23184b0) proc(0x22ed960) pid(14431) version 11:1:1:4 [ CSSD]2013-05-23 16:26:20.672 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22ed960), client(1/0x22ee090) [ CSSD]2013-05-23 16:26:20.672 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee090) [ CSSD]2013-05-23 16:26:20.672 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee090 with con 0x23192b0, requested num -1 [ CSSD]2013-05-23 16:26:20.672 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:20.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1490 (0x7f431bfed910) [ CSSD]2013-05-23 16:26:20.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed910 (RPC#1490) tag(5d2002c) sent to node 1 [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05d2002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014570, gin 1, member 0 [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed910 (RPC#1490) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfed910, ret 0, client 0x22ee090 member 0x22e3600 [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee090 [ CSSD]2013-05-23 16:26:20.673 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1490 [ CSSD]2013-05-23 16:26:20.679 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee090) [ CSSD]2013-05-23 16:26:20.679 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee090), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:20.679 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e3600) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:20.679 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1491 (0x7f431bfed990) [ CSSD]2013-05-23 16:26:20.679 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfed990 (RPC#1491) tag(5d3002c) sent to node 1 [ CSSD]2013-05-23 16:26:20.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05d3002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:20.680 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:20.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfed990 (RPC#1491) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:20.680 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfed990, ret 0, client 0x22ee090 member 0x22e3600 [ CSSD]2013-05-23 16:26:20.680 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1491 [ CSSD]2013-05-23 16:26:20.680 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e3600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:26:20.701 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:20.701 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:20.702 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:20.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:20.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:20.882 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed960 [ CSSD]2013-05-23 16:26:20.882 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed960) con(0x23184b0) skgpid ospid 14431 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:21.721 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:26:21.721 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1492 (0x7f431bfeda10) [ CSSD]2013-05-23 16:26:21.721 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeda10 (RPC#1492) tag(5d4002c) sent to node 1 [ CSSD]2013-05-23 16:26:21.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:21.722 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1492 (0x7f431bfeda10) [ CSSD]2013-05-23 16:26:21.722 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeda10 (RPC#1492) state 6, flags 0x202 [ CSSD]2013-05-23 16:26:21.722 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeda10, ret 0, client (nil) [ CSSD]2013-05-23 16:26:21.722 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1492 [ CSSD]2013-05-23 16:26:22.715 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:22.780 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:22.780 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:23.796 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:26:23.796 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:26:23.796 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:26:23.796 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:26:23.796 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134c40 [ CSSD]2013-05-23 16:26:23.796 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134c40) con(0x7f431c108e60) skgpid 9020 ospid 9020 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:24.720 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:24.832 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:24.832 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:25.704 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:25.704 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:25.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:25.708 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014571, gin 1, member 0 [ CSSD]2013-05-23 16:26:25.708 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2317f40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:26:25.708 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:26:25.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:25.715 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:26:25.715 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4fc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:26:26.726 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:26.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:26.837 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:28.639 [1191196992] >TRACE: clssgmAllocProc: (0x22ed960) allocated [ CSSD]2013-05-23 16:26:28.640 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:26:28.640 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed960 - 1,2,3 [ CSSD]2013-05-23 16:26:28.640 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23184b0) proc(0x22ed960) pid(14469/14469) version 11:1:1:4 [ CSSD]2013-05-23 16:26:28.646 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ed960), client(1/0x22edf00) [ CSSD]2013-05-23 16:26:28.646 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22edf00) [ CSSD]2013-05-23 16:26:28.646 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:26:28.646 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:26:28.737 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:28.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:28.843 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:30.583 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:26:30.584 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:26:30.585 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:26:30.585 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(14524) version 11:1:1:4 [ CSSD]2013-05-23 16:26:30.591 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:26:30.591 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:26:30.591 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:26:30.591 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:30.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1493 (0x7f431bfeda90) [ CSSD]2013-05-23 16:26:30.591 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeda90 (RPC#1493) tag(5d5002c) sent to node 1 [ CSSD]2013-05-23 16:26:30.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05d5002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:30.591 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014572, gin 1, member 0 [ CSSD]2013-05-23 16:26:30.591 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:26:30.592 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:26:30.592 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:26:30.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeda90 (RPC#1493) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:30.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeda90, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:26:30.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:26:30.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1493 [ CSSD]2013-05-23 16:26:30.598 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:26:30.598 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:30.598 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:30.598 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1494 (0x7f431bfedb10) [ CSSD]2013-05-23 16:26:30.598 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedb10 (RPC#1494) tag(5d6002c) sent to node 1 [ CSSD]2013-05-23 16:26:30.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05d6002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:30.598 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:30.599 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedb10 (RPC#1494) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:30.599 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfedb10, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:26:30.599 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1494 [ CSSD]2013-05-23 16:26:30.599 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:26:30.707 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:30.707 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:30.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:30.801 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:26:30.801 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 14524 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:30.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:30.854 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:31.746 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:26:31.747 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1495 (0x7f431bfedb90) [ CSSD]2013-05-23 16:26:31.747 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedb90 (RPC#1495) tag(5d7002c) sent to node 1 [ CSSD]2013-05-23 16:26:31.747 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:31.747 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1495 (0x7f431bfedb90) [ CSSD]2013-05-23 16:26:31.747 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedb90 (RPC#1495) state 6, flags 0x202 [ CSSD]2013-05-23 16:26:31.747 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfedb90, ret 0, client (nil) [ CSSD]2013-05-23 16:26:31.747 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1495 [ CSSD]2013-05-23 16:26:32.795 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:32.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:32.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:34.709 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:34.709 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:26:34.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:34.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:34.933 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:36.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:36.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:36.957 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:38.898 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:38.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:38.970 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:39.712 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:39.712 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:40.904 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:40.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:40.958 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014573, gin 1, member 0 [ CSSD]2013-05-23 16:26:40.958 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:26:40.958 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:26:40.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:40.965 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:26:40.965 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22edcf0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:26:40.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:40.975 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:41.842 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:26:41.843 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1496 (0x7f431bfedc10) [ CSSD]2013-05-23 16:26:41.843 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedc10 (RPC#1496) tag(5d8002c) sent to node 1 [ CSSD]2013-05-23 16:26:41.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:41.843 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1496 (0x7f431bfedc10) [ CSSD]2013-05-23 16:26:41.843 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedc10 (RPC#1496) state 6, flags 0x202 [ CSSD]2013-05-23 16:26:41.843 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfedc10, ret 0, client (nil) [ CSSD]2013-05-23 16:26:41.843 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1496 [ CSSD]2013-05-23 16:26:42.916 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:42.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:42.980 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:44.715 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:44.715 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:44.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:44.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:44.982 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:45.833 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:26:45.834 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:26:45.834 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:26:45.834 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(14705) version 11:1:1:4 [ CSSD]2013-05-23 16:26:45.850 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:26:45.850 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:26:45.850 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:26:45.850 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:45.850 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1497 (0x7f431bfedc90) [ CSSD]2013-05-23 16:26:45.850 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedc90 (RPC#1497) tag(5d9002c) sent to node 1 [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05d9002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014574, gin 1, member 0 [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedc90 (RPC#1497) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfedc90, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:26:45.851 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1497 [ CSSD]2013-05-23 16:26:45.858 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:26:45.858 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:45.858 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:26:45.858 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1498 (0x7f431bfedd10) [ CSSD]2013-05-23 16:26:45.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedd10 (RPC#1498) tag(5da002c) sent to node 1 [ CSSD]2013-05-23 16:26:45.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05da002c) incarnation(174631051) [ CSSD]2013-05-23 16:26:45.858 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:26:45.858 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedd10 (RPC#1498) state 6, flags 0x100 [ CSSD]2013-05-23 16:26:45.858 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfedd10, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:26:45.859 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1498 [ CSSD]2013-05-23 16:26:45.859 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:26:46.061 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:26:46.061 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 14705 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:26:46.927 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:47.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:47.008 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:48.938 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:49.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:49.040 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:49.718 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:49.718 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:50.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:51.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:51.046 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:51.868 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:26:51.869 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1499 (0x7f431bfedd90) [ CSSD]2013-05-23 16:26:51.869 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedd90 (RPC#1499) tag(5db002c) sent to node 1 [ CSSD]2013-05-23 16:26:51.870 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:51.870 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1499 (0x7f431bfedd90) [ CSSD]2013-05-23 16:26:51.870 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedd90 (RPC#1499) state 6, flags 0x202 [ CSSD]2013-05-23 16:26:51.870 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfedd90, ret 0, client (nil) [ CSSD]2013-05-23 16:26:51.870 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1499 [ CSSD]2013-05-23 16:26:52.996 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:53.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:53.052 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:54.721 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:54.721 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:26:55.002 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:55.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:55.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:56.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:56.223 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014575, gin 1, member 0 [ CSSD]2013-05-23 16:26:56.223 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:26:56.223 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:26:56.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:56.229 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:26:56.229 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:26:57.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:57.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:57.067 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:59.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:26:59.141 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:26:59.141 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:26:59.723 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:26:59.724 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:01.092 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:27:01.093 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:27:01.093 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:27:01.093 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(14842) version 11:1:1:4 [ CSSD]2013-05-23 16:27:01.100 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:27:01.100 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:01.100 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:27:01.100 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:01.100 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1500 (0x7f431bfede10) [ CSSD]2013-05-23 16:27:01.100 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfede10 (RPC#1500) tag(5dc002c) sent to node 1 [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05dc002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014576, gin 1, member 0 [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfede10 (RPC#1500) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfede10, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:27:01.101 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1500 [ CSSD]2013-05-23 16:27:01.104 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:01.108 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:01.108 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:01.108 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:01.108 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1501 (0x7f431bfede90) [ CSSD]2013-05-23 16:27:01.108 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfede90 (RPC#1501) tag(5dd002c) sent to node 1 [ CSSD]2013-05-23 16:27:01.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05dd002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:01.108 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:01.108 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfede90 (RPC#1501) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:01.108 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfede90, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:01.109 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1501 [ CSSD]2013-05-23 16:27:01.109 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:27:01.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:01.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:01.311 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:27:01.311 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 14842 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:01.895 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:27:01.895 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1502 (0x7f431bfedf10) [ CSSD]2013-05-23 16:27:01.895 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedf10 (RPC#1502) tag(5de002c) sent to node 1 [ CSSD]2013-05-23 16:27:01.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:01.896 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1502 (0x7f431bfedf10) [ CSSD]2013-05-23 16:27:01.896 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedf10 (RPC#1502) state 6, flags 0x202 [ CSSD]2013-05-23 16:27:01.896 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfedf10, ret 0, client (nil) [ CSSD]2013-05-23 16:27:01.896 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1502 [ CSSD]2013-05-23 16:27:03.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:03.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:03.172 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:04.726 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:04.727 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:05.123 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:05.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:05.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:07.128 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:07.189 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:07.189 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:09.139 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:09.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:09.241 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:09.729 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:09.729 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:11.144 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:11.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:11.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:11.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:11.478 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014577, gin 1, member 0 [ CSSD]2013-05-23 16:27:11.478 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:11.478 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:11.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:11.485 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:11.485 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:11.919 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:27:11.919 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1503 (0x7f431bfedf90) [ CSSD]2013-05-23 16:27:11.919 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfedf90 (RPC#1503) tag(5df002c) sent to node 1 [ CSSD]2013-05-23 16:27:11.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:11.920 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1503 (0x7f431bfedf90) [ CSSD]2013-05-23 16:27:11.920 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfedf90 (RPC#1503) state 6, flags 0x202 [ CSSD]2013-05-23 16:27:11.920 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfedf90, ret 0, client (nil) [ CSSD]2013-05-23 16:27:11.920 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1503 [ CSSD]2013-05-23 16:27:12.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:12.229 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014578, gin 1, member 0 [ CSSD]2013-05-23 16:27:12.229 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:12.229 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:12.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:12.236 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:12.236 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:13.197 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:13.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:13.252 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:14.732 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:14.732 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:15.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:15.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:15.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:16.082 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:27:16.083 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:27:16.084 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:27:16.084 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15193) version 11:1:1:4 [ CSSD]2013-05-23 16:27:16.203 [1191196992] >TRACE: clssgmAllocProc: (0x22e8320) allocated [ CSSD]2013-05-23 16:27:16.204 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:27:16.204 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8320 - 1,2,3 [ CSSD]2013-05-23 16:27:16.204 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321cb0) proc(0x22e8320) pid(15203) version 11:1:1:4 [ CSSD]2013-05-23 16:27:16.231 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:27:16.233 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:27:16.236 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8320 [ CSSD]2013-05-23 16:27:16.236 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8320) con(0x2321cb0) skgpid ospid 15203 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:16.306 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(68/0x2321ae0) [ CSSD]2013-05-23 16:27:16.306 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 68 (0x2321ae0) [ CSSD]2013-05-23 16:27:16.306 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x2321ae0 with con 0x22e4e00, requested num -1 [ CSSD]2013-05-23 16:27:16.306 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:27:16.306 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1504 (0x7f431bfee010) [ CSSD]2013-05-23 16:27:16.306 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee010 (RPC#1504) tag(5e0002c) sent to node 1 [ CSSD]2013-05-23 16:27:16.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05e0002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:16.306 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014579, gin 1, member 0 [ CSSD]2013-05-23 16:27:16.306 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x2321c50/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:27:16.307 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:27:16.307 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:27:16.307 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee010 (RPC#1504) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:16.307 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee010, ret 0, client 0x2321ae0 member 0x22e53d0 [ CSSD]2013-05-23 16:27:16.307 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ae0 [ CSSD]2013-05-23 16:27:16.307 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1504 [ CSSD]2013-05-23 16:27:16.312 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 68 (0x2321ae0) [ CSSD]2013-05-23 16:27:16.312 [1191196992] >TRACE: clssgmExitGrock: client 68 (0x2321ae0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:27:16.312 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e53d0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:27:16.312 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1505 (0x7f431bfee090) [ CSSD]2013-05-23 16:27:16.313 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee090 (RPC#1505) tag(5e1002c) sent to node 1 [ CSSD]2013-05-23 16:27:16.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05e1002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:16.313 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:27:16.313 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee090 (RPC#1505) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:16.313 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee090, ret 0, client 0x2321ae0 member 0x22e53d0 [ CSSD]2013-05-23 16:27:16.313 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1505 [ CSSD]2013-05-23 16:27:16.313 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22e53d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:27:16.314 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:27:16.314 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15193 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:16.344 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:27:16.346 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:27:16.346 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:27:16.346 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15213) version 11:1:1:4 [ CSSD]2013-05-23 16:27:16.362 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:27:16.362 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:16.362 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:27:16.362 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:16.362 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1506 (0x7f431bfee110) [ CSSD]2013-05-23 16:27:16.362 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee110 (RPC#1506) tag(5e2002c) sent to node 1 [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05e2002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014580, gin 1, member 0 [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee110 (RPC#1506) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee110, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:27:16.363 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1506 [ CSSD]2013-05-23 16:27:16.369 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:16.369 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:16.369 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:16.369 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1507 (0x7f431bfee190) [ CSSD]2013-05-23 16:27:16.369 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee190 (RPC#1507) tag(5e3002c) sent to node 1 [ CSSD]2013-05-23 16:27:16.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05e3002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:16.369 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:16.369 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee190 (RPC#1507) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:16.369 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee190, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:16.370 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1507 [ CSSD]2013-05-23 16:27:16.370 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:27:16.572 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:27:16.572 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15213 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:17.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:17.120 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014581, gin 1, member 0 [ CSSD]2013-05-23 16:27:17.120 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:17.120 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:17.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:17.126 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:17.126 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:17.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:17.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:17.270 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:19.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:19.342 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:19.342 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:19.735 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:19.735 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:20.914 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:27:20.915 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:27:20.915 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:27:20.915 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15278) version 11:1:1:4 [ CSSD]2013-05-23 16:27:20.932 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:27:20.932 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:20.932 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:27:20.932 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:20.932 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1508 (0x7f431bfee210) [ CSSD]2013-05-23 16:27:20.932 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee210 (RPC#1508) tag(5e4002c) sent to node 1 [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05e4002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014582, gin 1, member 0 [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee210 (RPC#1508) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee210, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:27:20.933 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1508 [ CSSD]2013-05-23 16:27:20.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:20.939 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:20.939 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:20.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1509 (0x7f431bfee290) [ CSSD]2013-05-23 16:27:20.940 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee290 (RPC#1509) tag(5e5002c) sent to node 1 [ CSSD]2013-05-23 16:27:20.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05e5002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:20.940 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:20.940 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee290 (RPC#1509) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:20.940 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee290, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:20.940 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1509 [ CSSD]2013-05-23 16:27:20.940 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:27:21.143 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:27:21.143 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15278 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:21.225 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:21.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:21.368 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:21.944 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:27:21.945 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1510 (0x7f431bfee310) [ CSSD]2013-05-23 16:27:21.945 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee310 (RPC#1510) tag(5e6002c) sent to node 1 [ CSSD]2013-05-23 16:27:21.945 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:21.945 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1510 (0x7f431bfee310) [ CSSD]2013-05-23 16:27:21.945 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee310 (RPC#1510) state 6, flags 0x202 [ CSSD]2013-05-23 16:27:21.945 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfee310, ret 0, client (nil) [ CSSD]2013-05-23 16:27:21.945 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1510 [ CSSD]2013-05-23 16:27:23.238 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:23.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:23.374 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:24.738 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:24.738 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:25.243 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:25.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:25.385 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:26.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:26.729 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014583, gin 1, member 0 [ CSSD]2013-05-23 16:27:26.729 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:26.729 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:26.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:26.735 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:26.735 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:27.249 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:27.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:27.391 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:29.270 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:29.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:29.443 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:29.741 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:29.741 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:31.276 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:31.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:31.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:31.605 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:27:31.606 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:27:31.607 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:27:31.607 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15356) version 11:1:1:4 [ CSSD]2013-05-23 16:27:31.613 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:27:31.613 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:31.613 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:27:31.613 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:31.613 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1511 (0x7f431bfee390) [ CSSD]2013-05-23 16:27:31.613 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee390 (RPC#1511) tag(5e7002c) sent to node 1 [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05e7002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014584, gin 1, member 0 [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee390 (RPC#1511) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee390, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:27:31.614 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1511 [ CSSD]2013-05-23 16:27:31.621 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:31.621 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:31.621 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:31.621 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1512 (0x7f431bfee410) [ CSSD]2013-05-23 16:27:31.621 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee410 (RPC#1512) tag(5e8002c) sent to node 1 [ CSSD]2013-05-23 16:27:31.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05e8002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:31.621 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:31.621 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee410 (RPC#1512) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:31.621 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee410, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:31.621 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1512 [ CSSD]2013-05-23 16:27:31.621 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:27:31.824 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:27:31.824 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15356 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:31.971 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:27:31.971 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1513 (0x7f431bfee490) [ CSSD]2013-05-23 16:27:31.971 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee490 (RPC#1513) tag(5e9002c) sent to node 1 [ CSSD]2013-05-23 16:27:31.972 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:31.972 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1513 (0x7f431bfee490) [ CSSD]2013-05-23 16:27:31.972 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee490 (RPC#1513) state 6, flags 0x202 [ CSSD]2013-05-23 16:27:31.972 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfee490, ret 0, client (nil) [ CSSD]2013-05-23 16:27:31.972 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1513 [ CSSD]2013-05-23 16:27:33.307 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:33.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:33.496 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:34.744 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:34.744 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:35.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:35.546 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:35.546 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:37.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:37.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:37.552 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:39.330 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:39.562 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:39.562 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:39.747 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:39.747 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:41.336 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:41.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:41.589 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:41.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:41.738 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014585, gin 1, member 0 [ CSSD]2013-05-23 16:27:41.738 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:41.738 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:41.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:41.745 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:41.745 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:41.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:41.979 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014586, gin 1, member 0 [ CSSD]2013-05-23 16:27:41.979 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:41.979 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:41.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:41.986 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:41.986 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:41.995 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:27:41.995 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1514 (0x7f431bfee510) [ CSSD]2013-05-23 16:27:41.995 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee510 (RPC#1514) tag(5ea002c) sent to node 1 [ CSSD]2013-05-23 16:27:41.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:41.996 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1514 (0x7f431bfee510) [ CSSD]2013-05-23 16:27:41.996 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee510 (RPC#1514) state 6, flags 0x202 [ CSSD]2013-05-23 16:27:41.996 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfee510, ret 0, client (nil) [ CSSD]2013-05-23 16:27:41.996 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1514 [ CSSD]2013-05-23 16:27:43.428 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:43.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:43.594 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:43.750 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:43.750 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:27:45.434 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:45.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:45.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:46.865 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:27:46.865 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:27:46.866 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:27:46.866 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15570) version 11:1:1:4 [ CSSD]2013-05-23 16:27:46.882 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:27:46.882 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:46.882 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:27:46.882 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:46.882 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1515 (0x7f431bfee590) [ CSSD]2013-05-23 16:27:46.882 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee590 (RPC#1515) tag(5eb002c) sent to node 1 [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05eb002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014587, gin 1, member 0 [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee590 (RPC#1515) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee590, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:27:46.883 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1515 [ CSSD]2013-05-23 16:27:46.890 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:27:46.890 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:46.890 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:27:46.890 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1516 (0x7f431bfee610) [ CSSD]2013-05-23 16:27:46.890 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee610 (RPC#1516) tag(5ec002c) sent to node 1 [ CSSD]2013-05-23 16:27:46.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05ec002c) incarnation(174631051) [ CSSD]2013-05-23 16:27:46.891 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:27:46.891 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee610 (RPC#1516) state 6, flags 0x100 [ CSSD]2013-05-23 16:27:46.891 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee610, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:27:46.891 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1516 [ CSSD]2013-05-23 16:27:46.891 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:27:47.093 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:27:47.093 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15570 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:27:47.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:47.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:47.652 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:48.753 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:48.753 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:49.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:49.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:49.664 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:51.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:51.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:51.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:52.020 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:27:52.020 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1517 (0x7f431bfee690) [ CSSD]2013-05-23 16:27:52.021 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee690 (RPC#1517) tag(5ed002c) sent to node 1 [ CSSD]2013-05-23 16:27:52.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:52.021 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1517 (0x7f431bfee690) [ CSSD]2013-05-23 16:27:52.021 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee690 (RPC#1517) state 6, flags 0x202 [ CSSD]2013-05-23 16:27:52.021 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfee690, ret 0, client (nil) [ CSSD]2013-05-23 16:27:52.021 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1517 [ CSSD]2013-05-23 16:27:53.548 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:53.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:53.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:53.755 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:53.756 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:55.553 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:55.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:55.686 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:57.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:57.228 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014588, gin 1, member 0 [ CSSD]2013-05-23 16:27:57.228 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:27:57.228 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:27:57.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:57.235 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:27:57.235 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:27:57.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:57.692 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:57.692 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:27:58.758 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:27:58.758 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:27:59.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:27:59.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:27:59.745 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:01.655 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:01.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:01.750 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:02.047 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:28:02.047 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1518 (0x7f431bfee710) [ CSSD]2013-05-23 16:28:02.048 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee710 (RPC#1518) tag(5ee002c) sent to node 1 [ CSSD]2013-05-23 16:28:02.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:02.048 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1518 (0x7f431bfee710) [ CSSD]2013-05-23 16:28:02.048 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee710 (RPC#1518) state 6, flags 0x202 [ CSSD]2013-05-23 16:28:02.048 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfee710, ret 0, client (nil) [ CSSD]2013-05-23 16:28:02.048 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1518 [ CSSD]2013-05-23 16:28:02.125 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:28:02.126 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:28:02.127 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:28:02.127 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15747) version 11:1:1:4 [ CSSD]2013-05-23 16:28:02.143 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:28:02.144 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:02.144 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:28:02.144 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:02.144 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1519 (0x7f431bfee790) [ CSSD]2013-05-23 16:28:02.144 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee790 (RPC#1519) tag(5ef002c) sent to node 1 [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05ef002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014589, gin 1, member 0 [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee790 (RPC#1519) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee790, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:28:02.144 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1519 [ CSSD]2013-05-23 16:28:02.151 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:02.151 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:02.151 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:02.151 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1520 (0x7f431bfee810) [ CSSD]2013-05-23 16:28:02.152 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee810 (RPC#1520) tag(5f0002c) sent to node 1 [ CSSD]2013-05-23 16:28:02.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05f0002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:02.152 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:02.152 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee810 (RPC#1520) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:02.152 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee810, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:02.152 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1520 [ CSSD]2013-05-23 16:28:02.152 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:28:02.354 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:28:02.354 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15747 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:03.709 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:03.761 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:03.761 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:03.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:03.776 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:05.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:05.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:05.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:07.727 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:07.764 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:07.764 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:28:07.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:07.793 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:09.732 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:09.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:09.815 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:11.738 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:11.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:11.821 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:12.074 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:28:12.074 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1521 (0x7f431bfee890) [ CSSD]2013-05-23 16:28:12.075 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee890 (RPC#1521) tag(5f1002c) sent to node 1 [ CSSD]2013-05-23 16:28:12.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:12.075 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1521 (0x7f431bfee890) [ CSSD]2013-05-23 16:28:12.075 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee890 (RPC#1521) state 6, flags 0x202 [ CSSD]2013-05-23 16:28:12.075 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfee890, ret 0, client (nil) [ CSSD]2013-05-23 16:28:12.075 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1521 [ CSSD]2013-05-23 16:28:12.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:12.477 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014590, gin 1, member 0 [ CSSD]2013-05-23 16:28:12.478 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:28:12.478 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:28:12.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:12.485 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:28:12.485 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:28:12.767 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:12.767 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:13.750 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:13.827 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:13.827 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:15.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:15.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:15.858 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:17.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:17.384 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014591, gin 1, member 0 [ CSSD]2013-05-23 16:28:17.384 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:28:17.384 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:28:17.392 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:17.392 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:28:17.392 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:28:17.393 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:28:17.394 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:28:17.395 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:28:17.395 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15896) version 11:1:1:4 [ CSSD]2013-05-23 16:28:17.411 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:28:17.411 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:17.411 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:28:17.411 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:17.411 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1522 (0x7f431bfee910) [ CSSD]2013-05-23 16:28:17.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee910 (RPC#1522) tag(5f2002c) sent to node 1 [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05f2002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014592, gin 1, member 0 [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee910 (RPC#1522) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfee910, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:28:17.411 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1522 [ CSSD]2013-05-23 16:28:17.418 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:17.418 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:17.418 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:17.418 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1523 (0x7f431bfee990) [ CSSD]2013-05-23 16:28:17.418 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfee990 (RPC#1523) tag(5f3002c) sent to node 1 [ CSSD]2013-05-23 16:28:17.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05f3002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:17.418 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:17.418 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfee990 (RPC#1523) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:17.418 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfee990, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:17.419 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1523 [ CSSD]2013-05-23 16:28:17.419 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:28:17.621 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:28:17.621 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15896 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:17.770 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:17.770 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:17.807 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:17.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:17.883 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:19.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:19.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:19.894 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:21.187 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:28:21.188 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:28:21.189 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:28:21.189 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(15961) version 11:1:1:4 [ CSSD]2013-05-23 16:28:21.195 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:28:21.195 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:21.195 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:28:21.195 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:21.195 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1524 (0x7f431bfeea10) [ CSSD]2013-05-23 16:28:21.195 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeea10 (RPC#1524) tag(5f4002c) sent to node 1 [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05f4002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014593, gin 1, member 0 [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeea10 (RPC#1524) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeea10, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:28:21.196 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1524 [ CSSD]2013-05-23 16:28:21.202 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:21.202 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:21.202 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:21.202 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1525 (0x7f431bfeea90) [ CSSD]2013-05-23 16:28:21.202 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeea90 (RPC#1525) tag(5f5002c) sent to node 1 [ CSSD]2013-05-23 16:28:21.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05f5002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:21.202 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:21.202 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeea90 (RPC#1525) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:21.202 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeea90, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:21.203 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1525 [ CSSD]2013-05-23 16:28:21.203 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:28:21.405 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:28:21.405 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 15961 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:21.818 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:21.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:21.899 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:22.099 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:28:22.099 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1526 (0x7f431bfeeb10) [ CSSD]2013-05-23 16:28:22.099 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeeb10 (RPC#1526) tag(5f6002c) sent to node 1 [ CSSD]2013-05-23 16:28:22.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:22.100 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1526 (0x7f431bfeeb10) [ CSSD]2013-05-23 16:28:22.100 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeeb10 (RPC#1526) state 6, flags 0x202 [ CSSD]2013-05-23 16:28:22.100 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeeb10, ret 0, client (nil) [ CSSD]2013-05-23 16:28:22.100 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1526 [ CSSD]2013-05-23 16:28:22.772 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:22.773 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:23.890 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:23.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:23.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:25.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:25.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:25.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:26.775 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:26.775 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:28:27.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:27.724 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014594, gin 1, member 0 [ CSSD]2013-05-23 16:28:27.724 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:28:27.724 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:28:27.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:27.731 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:28:27.731 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee160) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:28:27.908 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:27.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:27.984 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:29.914 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:29.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:29.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:31.778 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:31.778 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:31.920 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:32.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:32.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:32.123 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:28:32.124 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1527 (0x7f431bfeeb90) [ CSSD]2013-05-23 16:28:32.124 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeeb90 (RPC#1527) tag(5f7002c) sent to node 1 [ CSSD]2013-05-23 16:28:32.125 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:32.125 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1527 (0x7f431bfeeb90) [ CSSD]2013-05-23 16:28:32.125 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeeb90 (RPC#1527) state 6, flags 0x202 [ CSSD]2013-05-23 16:28:32.125 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeeb90, ret 0, client (nil) [ CSSD]2013-05-23 16:28:32.125 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1527 [ CSSD]2013-05-23 16:28:32.656 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:28:32.657 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:28:32.658 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:28:32.658 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(16048) version 11:1:1:4 [ CSSD]2013-05-23 16:28:32.674 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x22e5460) [ CSSD]2013-05-23 16:28:32.674 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:32.674 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:28:32.674 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:32.674 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1528 (0x7f431bfeec10) [ CSSD]2013-05-23 16:28:32.675 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeec10 (RPC#1528) tag(5f8002c) sent to node 1 [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05f8002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014595, gin 1, member 0 [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeec10 (RPC#1528) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeec10, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:32.675 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:28:32.676 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1528 [ CSSD]2013-05-23 16:28:32.682 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:28:32.682 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:32.682 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:32.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1529 (0x7f431bfeec90) [ CSSD]2013-05-23 16:28:32.682 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeec90 (RPC#1529) tag(5f9002c) sent to node 1 [ CSSD]2013-05-23 16:28:32.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05f9002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:32.682 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:32.682 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeec90 (RPC#1529) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:32.682 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeec90, ret 0, client 0x22e5460 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:32.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1529 [ CSSD]2013-05-23 16:28:32.683 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:28:32.885 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:28:32.885 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 16048 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:33.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:34.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:34.054 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:35.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:36.079 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:36.079 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:36.781 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:36.781 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:38.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:38.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:38.085 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:38.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:38.755 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014596, gin 1, member 0 [ CSSD]2013-05-23 16:28:38.755 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:28:38.755 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:28:38.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:38.761 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:28:38.761 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c1509e0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:28:38.886 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:28:38.887 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:28:38.888 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:28:38.889 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:28:38.890 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:28:38.890 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:28:40.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:40.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:40.096 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:41.784 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:41.784 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:42.040 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:42.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:42.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:42.149 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:28:42.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1530 (0x7f431bfeed10) [ CSSD]2013-05-23 16:28:42.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeed10 (RPC#1530) tag(5fa002c) sent to node 1 [ CSSD]2013-05-23 16:28:42.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:42.150 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1530 (0x7f431bfeed10) [ CSSD]2013-05-23 16:28:42.150 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeed10 (RPC#1530) state 6, flags 0x202 [ CSSD]2013-05-23 16:28:42.150 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeed10, ret 0, client (nil) [ CSSD]2013-05-23 16:28:42.150 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1530 [ CSSD]2013-05-23 16:28:42.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:42.983 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014597, gin 1, member 0 [ CSSD]2013-05-23 16:28:42.984 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:28:42.984 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:28:42.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:42.989 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:28:42.989 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1509e0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:28:44.051 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:44.154 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:44.154 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:46.056 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:46.160 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:46.160 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:46.787 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:46.787 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:47.917 [1191196992] >TRACE: clssgmAllocProc: (0x22e5010) allocated [ CSSD]2013-05-23 16:28:47.918 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:28:47.918 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5010 - 1,2,3 [ CSSD]2013-05-23 16:28:47.918 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5010) pid(16188) version 11:1:1:4 [ CSSD]2013-05-23 16:28:47.925 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e5010), client(1/0x2317d30) [ CSSD]2013-05-23 16:28:47.925 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:28:47.925 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317d30 with con 0x22e52e0, requested num -1 [ CSSD]2013-05-23 16:28:47.925 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:47.925 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1531 (0x7f431bfeed90) [ CSSD]2013-05-23 16:28:47.925 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeed90 (RPC#1531) tag(5fb002c) sent to node 1 [ CSSD]2013-05-23 16:28:47.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05fb002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:47.925 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014598, gin 1, member 0 [ CSSD]2013-05-23 16:28:47.925 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:28:47.926 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:28:47.926 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:28:47.926 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeed90 (RPC#1531) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:47.926 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeed90, ret 0, client 0x2317d30 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:47.926 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317d30 [ CSSD]2013-05-23 16:28:47.926 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1531 [ CSSD]2013-05-23 16:28:47.933 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:28:47.933 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:47.933 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:28:47.933 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1532 (0x7f431bfeee10) [ CSSD]2013-05-23 16:28:47.933 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeee10 (RPC#1532) tag(5fc002c) sent to node 1 [ CSSD]2013-05-23 16:28:47.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(05fc002c) incarnation(174631051) [ CSSD]2013-05-23 16:28:47.934 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:28:47.934 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeee10 (RPC#1532) state 6, flags 0x100 [ CSSD]2013-05-23 16:28:47.934 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeee10, ret 0, client 0x2317d30 member 0x22e4ef0 [ CSSD]2013-05-23 16:28:47.934 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1532 [ CSSD]2013-05-23 16:28:47.934 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:28:48.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:48.136 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5010 [ CSSD]2013-05-23 16:28:48.136 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5010) con(0x22e36e0) skgpid ospid 16188 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:48.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:48.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:50.062 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:50.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:50.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:50.789 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:50.789 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:28:52.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:52.173 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:28:52.173 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1533 (0x7f431bfeee90) [ CSSD]2013-05-23 16:28:52.173 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeee90 (RPC#1533) tag(5fd002c) sent to node 1 [ CSSD]2013-05-23 16:28:52.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:52.174 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1533 (0x7f431bfeee90) [ CSSD]2013-05-23 16:28:52.174 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeee90 (RPC#1533) state 6, flags 0x202 [ CSSD]2013-05-23 16:28:52.174 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeee90, ret 0, client (nil) [ CSSD]2013-05-23 16:28:52.174 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1533 [ CSSD]2013-05-23 16:28:52.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:52.200 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:53.832 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 16:28:53.832 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c137e70), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:28:53.832 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c137e70), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:28:53.832 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:28:53.832 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c147ff0 [ CSSD]2013-05-23 16:28:53.832 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c147ff0) con(0x7f431c14e8d0) skgpid 10916 ospid 10916 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:53.836 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c137bd0) [ CSSD]2013-05-23 16:28:53.836 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c137bd0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:28:53.836 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c137bd0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:28:53.837 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:28:53.846 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c1377b0) [ CSSD]2013-05-23 16:28:53.846 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c1377b0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:28:53.846 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c1377b0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:28:53.846 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:28:53.846 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838b0 [ CSSD]2013-05-23 16:28:53.846 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838b0) con(0x7f431c0829d0) skgpid 10926 ospid 10926 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:28:54.121 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:54.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:54.255 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:55.792 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:28:55.792 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:28:56.127 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:56.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:56.261 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:58.139 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:28:58.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:58.241 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014599, gin 1, member 0 [ CSSD]2013-05-23 16:28:58.241 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:28:58.241 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:28:58.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:58.247 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:28:58.247 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:28:58.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:28:58.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:28:58.336 [1191196992] >TRACE: clssgmAllocProc: (0x22e5060) allocated [ CSSD]2013-05-23 16:28:58.337 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:28:58.338 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5060 - 1,2,3 [ CSSD]2013-05-23 16:28:58.338 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5060) pid(16276/16276) version 11:1:1:4 [ CSSD]2013-05-23 16:28:58.344 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e5060), client(1/0x2317d30) [ CSSD]2013-05-23 16:28:58.344 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:28:58.344 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:28:58.344 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:28:58.738 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e5060), client(2/0x2321dd0) [ CSSD]2013-05-23 16:28:58.738 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2321dd0) [ CSSD]2013-05-23 16:28:58.738 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:28:58.738 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:29:00.145 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:00.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:00.297 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:00.795 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:00.795 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:02.151 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:02.197 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:29:02.197 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1534 (0x7f431bfeef10) [ CSSD]2013-05-23 16:29:02.197 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeef10 (RPC#1534) tag(5fe002c) sent to node 1 [ CSSD]2013-05-23 16:29:02.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:02.198 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1534 (0x7f431bfeef10) [ CSSD]2013-05-23 16:29:02.198 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeef10 (RPC#1534) state 6, flags 0x202 [ CSSD]2013-05-23 16:29:02.198 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfeef10, ret 0, client (nil) [ CSSD]2013-05-23 16:29:02.198 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1534 [ CSSD]2013-05-23 16:29:02.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:02.303 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:03.178 [1191196992] >TRACE: clssgmAllocProc: (0x22e8bd0) allocated [ CSSD]2013-05-23 16:29:03.179 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:29:03.179 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8bd0 - 1,2,3 [ CSSD]2013-05-23 16:29:03.179 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23224d0) proc(0x22e8bd0) pid(16341) version 11:1:1:4 [ CSSD]2013-05-23 16:29:03.197 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8bd0), client(1/0x22e9290) [ CSSD]2013-05-23 16:29:03.197 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e9290) [ CSSD]2013-05-23 16:29:03.197 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e9290 with con 0x2323150, requested num -1 [ CSSD]2013-05-23 16:29:03.197 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:03.197 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1535 (0x7f431bfeef90) [ CSSD]2013-05-23 16:29:03.197 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeef90 (RPC#1535) tag(5ff002c) sent to node 1 [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(05ff002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014600, gin 1, member 0 [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9400/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeef90 (RPC#1535) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeef90, ret 0, client 0x22e9290 member 0x22e8ab0 [ CSSD]2013-05-23 16:29:03.197 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e9290 [ CSSD]2013-05-23 16:29:03.198 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1535 [ CSSD]2013-05-23 16:29:03.203 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e9290) [ CSSD]2013-05-23 16:29:03.203 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e9290), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:03.203 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8ab0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:03.203 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1536 (0x7f431bfef010) [ CSSD]2013-05-23 16:29:03.204 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef010 (RPC#1536) tag(600002c) sent to node 1 [ CSSD]2013-05-23 16:29:03.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0600002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:03.204 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:03.204 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef010 (RPC#1536) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:03.204 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef010, ret 0, client 0x22e9290 member 0x22e8ab0 [ CSSD]2013-05-23 16:29:03.204 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1536 [ CSSD]2013-05-23 16:29:03.204 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8ab0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:29:03.407 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8bd0 [ CSSD]2013-05-23 16:29:03.407 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8bd0) con(0x23224d0) skgpid ospid 16341 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:29:04.162 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:04.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:04.355 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:05.799 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:05.799 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:06.168 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:06.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:06.360 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:08.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:08.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:08.366 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:10.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:10.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:10.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:10.802 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:10.802 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:12.229 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:29:12.229 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1537 (0x7f431bfef090) [ CSSD]2013-05-23 16:29:12.229 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef090 (RPC#1537) tag(601002c) sent to node 1 [ CSSD]2013-05-23 16:29:12.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:12.230 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1537 (0x7f431bfef090) [ CSSD]2013-05-23 16:29:12.230 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef090 (RPC#1537) state 6, flags 0x202 [ CSSD]2013-05-23 16:29:12.230 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef090, ret 0, client (nil) [ CSSD]2013-05-23 16:29:12.230 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1537 [ CSSD]2013-05-23 16:29:12.230 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:12.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:12.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:13.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:13.490 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014601, gin 1, member 0 [ CSSD]2013-05-23 16:29:13.490 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:29:13.490 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:29:13.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:13.495 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:29:13.495 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:29:14.242 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:14.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:14.456 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:15.805 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:15.805 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:16.248 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:16.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:16.481 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:17.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:17.647 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014602, gin 1, member 0 [ CSSD]2013-05-23 16:29:17.647 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:29:17.647 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:29:17.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:17.653 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:29:17.653 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:29:18.259 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:18.438 [1191196992] >TRACE: clssgmAllocProc: (0x22e8ca0) allocated [ CSSD]2013-05-23 16:29:18.439 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:29:18.439 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8ca0 - 1,2,3 [ CSSD]2013-05-23 16:29:18.439 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2322600) proc(0x22e8ca0) pid(16498) version 11:1:1:4 [ CSSD]2013-05-23 16:29:18.446 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8ca0), client(1/0x23231b0) [ CSSD]2013-05-23 16:29:18.446 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23231b0) [ CSSD]2013-05-23 16:29:18.446 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23231b0 with con 0x22e8f70, requested num -1 [ CSSD]2013-05-23 16:29:18.446 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:18.446 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1538 (0x7f431bfef110) [ CSSD]2013-05-23 16:29:18.446 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef110 (RPC#1538) tag(602002c) sent to node 1 [ CSSD]2013-05-23 16:29:18.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0602002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:18.446 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014603, gin 1, member 0 [ CSSD]2013-05-23 16:29:18.446 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9550/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:29:18.447 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:29:18.447 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:29:18.447 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef110 (RPC#1538) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:18.447 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfef110, ret 0, client 0x23231b0 member 0x22e8b80 [ CSSD]2013-05-23 16:29:18.447 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23231b0 [ CSSD]2013-05-23 16:29:18.447 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1538 [ CSSD]2013-05-23 16:29:18.453 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23231b0) [ CSSD]2013-05-23 16:29:18.453 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23231b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:18.453 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8b80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:18.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1539 (0x7f431bfef190) [ CSSD]2013-05-23 16:29:18.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef190 (RPC#1539) tag(603002c) sent to node 1 [ CSSD]2013-05-23 16:29:18.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0603002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:18.454 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:18.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef190 (RPC#1539) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:18.454 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef190, ret 0, client 0x23231b0 member 0x22e8b80 [ CSSD]2013-05-23 16:29:18.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1539 [ CSSD]2013-05-23 16:29:18.454 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8b80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:29:18.494 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:18.494 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:18.656 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8ca0 [ CSSD]2013-05-23 16:29:18.656 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8ca0) con(0x2322600) skgpid ospid 16498 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:29:20.264 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:20.498 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:20.498 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:20.808 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:20.808 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:21.447 [1191196992] >TRACE: clssgmAllocProc: (0x22e8bd0) allocated [ CSSD]2013-05-23 16:29:21.448 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:29:21.449 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8bd0 - 1,2,3 [ CSSD]2013-05-23 16:29:21.449 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321f40) proc(0x22e8bd0) pid(16557) version 11:1:1:4 [ CSSD]2013-05-23 16:29:21.455 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8bd0), client(1/0x22e9300) [ CSSD]2013-05-23 16:29:21.455 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e9300) [ CSSD]2013-05-23 16:29:21.455 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e9300 with con 0x2323170, requested num -1 [ CSSD]2013-05-23 16:29:21.455 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:21.455 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1540 (0x7f431bfef210) [ CSSD]2013-05-23 16:29:21.455 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef210 (RPC#1540) tag(604002c) sent to node 1 [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0604002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014604, gin 1, member 0 [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9470/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef210 (RPC#1540) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfef210, ret 0, client 0x22e9300 member 0x22e8ab0 [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e9300 [ CSSD]2013-05-23 16:29:21.456 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1540 [ CSSD]2013-05-23 16:29:21.462 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e9300) [ CSSD]2013-05-23 16:29:21.462 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e9300), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:21.462 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8ab0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:21.462 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1541 (0x7f431bfef290) [ CSSD]2013-05-23 16:29:21.463 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef290 (RPC#1541) tag(605002c) sent to node 1 [ CSSD]2013-05-23 16:29:21.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0605002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:21.463 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:21.463 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef290 (RPC#1541) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:21.463 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef290, ret 0, client 0x22e9300 member 0x22e8ab0 [ CSSD]2013-05-23 16:29:21.463 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1541 [ CSSD]2013-05-23 16:29:21.463 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8ab0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:29:21.666 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8bd0 [ CSSD]2013-05-23 16:29:21.666 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8bd0) con(0x2321f40) skgpid ospid 16557 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:29:22.267 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:29:22.267 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1542 (0x7f431bfef310) [ CSSD]2013-05-23 16:29:22.268 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef310 (RPC#1542) tag(606002c) sent to node 1 [ CSSD]2013-05-23 16:29:22.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:22.268 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1542 (0x7f431bfef310) [ CSSD]2013-05-23 16:29:22.268 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef310 (RPC#1542) state 6, flags 0x202 [ CSSD]2013-05-23 16:29:22.268 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef310, ret 0, client (nil) [ CSSD]2013-05-23 16:29:22.268 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1542 [ CSSD]2013-05-23 16:29:22.269 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:22.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:22.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:24.322 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:24.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:24.556 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:25.706 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:29:25.706 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:29:25.707 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:29:25.708 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:29:25.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:25.791 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014605, gin 1, member 0 [ CSSD]2013-05-23 16:29:25.791 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c1215d0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:29:25.791 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:29:25.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:25.799 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:29:25.799 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:29:25.811 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:25.811 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:26.327 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:26.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:26.582 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:28.420 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:28.588 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:28.588 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:28.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:28.739 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014606, gin 1, member 0 [ CSSD]2013-05-23 16:29:28.739 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:29:28.739 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:29:28.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:28.745 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:29:28.745 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:29:30.426 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:30.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:30.599 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:30.814 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:30.814 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:32.294 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:29:32.294 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1543 (0x7f431bfef390) [ CSSD]2013-05-23 16:29:32.294 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef390 (RPC#1543) tag(607002c) sent to node 1 [ CSSD]2013-05-23 16:29:32.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:32.295 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1543 (0x7f431bfef390) [ CSSD]2013-05-23 16:29:32.295 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef390 (RPC#1543) state 6, flags 0x202 [ CSSD]2013-05-23 16:29:32.295 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef390, ret 0, client (nil) [ CSSD]2013-05-23 16:29:32.295 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1543 [ CSSD]2013-05-23 16:29:32.432 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:32.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:32.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:33.443 [1191196992] >TRACE: clssgmAllocProc: (0x22e8bd0) allocated [ CSSD]2013-05-23 16:29:33.444 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:29:33.444 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8bd0 - 1,2,3 [ CSSD]2013-05-23 16:29:33.444 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23224d0) proc(0x22e8bd0) pid(15653/15653) version 11:1:1:4 [ CSSD]2013-05-23 16:29:33.461 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8bd0), client(1/0x22e9100) [ CSSD]2013-05-23 16:29:33.461 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e9100) [ CSSD]2013-05-23 16:29:33.461 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:29:33.461 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:29:33.688 [1191196992] >TRACE: clssgmAllocProc: (0x2327a60) allocated [ CSSD]2013-05-23 16:29:33.689 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:29:33.689 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327a60 - 1,2,3 [ CSSD]2013-05-23 16:29:33.689 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e97e0) proc(0x2327a60) pid(16660) version 11:1:1:4 [ CSSD]2013-05-23 16:29:33.706 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2327a60), client(1/0x2328220) [ CSSD]2013-05-23 16:29:33.706 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2328220) [ CSSD]2013-05-23 16:29:33.706 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2328220 with con 0x2327d10, requested num -1 [ CSSD]2013-05-23 16:29:33.706 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:33.706 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1544 (0x7f431bfef410) [ CSSD]2013-05-23 16:29:33.706 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef410 (RPC#1544) tag(608002c) sent to node 1 [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0608002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014607, gin 1, member 0 [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328390/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef410 (RPC#1544) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfef410, ret 0, client 0x2328220 member 0x2327940 [ CSSD]2013-05-23 16:29:33.706 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2328220 [ CSSD]2013-05-23 16:29:33.707 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1544 [ CSSD]2013-05-23 16:29:33.712 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2328220) [ CSSD]2013-05-23 16:29:33.712 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2328220), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:33.712 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:33.712 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1545 (0x7f431bfef490) [ CSSD]2013-05-23 16:29:33.712 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef490 (RPC#1545) tag(609002c) sent to node 1 [ CSSD]2013-05-23 16:29:33.713 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0609002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:33.713 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:33.713 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef490 (RPC#1545) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:33.713 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef490, ret 0, client 0x2328220 member 0x2327940 [ CSSD]2013-05-23 16:29:33.713 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1545 [ CSSD]2013-05-23 16:29:33.713 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:29:33.915 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327a60 [ CSSD]2013-05-23 16:29:33.915 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327a60) con(0x22e97e0) skgpid ospid 16660 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:29:34.443 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:34.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:34.625 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:34.816 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:34.817 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:29:36.448 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:36.631 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:36.631 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:38.461 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:38.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:38.638 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:39.819 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:39.819 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:40.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:40.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:40.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:42.320 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:29:42.321 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1546 (0x7f431bfef510) [ CSSD]2013-05-23 16:29:42.321 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef510 (RPC#1546) tag(60a002c) sent to node 1 [ CSSD]2013-05-23 16:29:42.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:42.321 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1546 (0x7f431bfef510) [ CSSD]2013-05-23 16:29:42.321 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef510 (RPC#1546) state 6, flags 0x202 [ CSSD]2013-05-23 16:29:42.321 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef510, ret 0, client (nil) [ CSSD]2013-05-23 16:29:42.321 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1546 [ CSSD]2013-05-23 16:29:42.518 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:42.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:42.695 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:43.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:43.988 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014608, gin 1, member 0 [ CSSD]2013-05-23 16:29:43.988 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:29:43.988 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:29:43.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:43.994 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:29:43.994 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:29:44.564 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:44.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:44.786 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:44.822 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:44.823 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:46.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:46.792 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:46.792 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:48.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:48.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:48.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:48.948 [1191196992] >TRACE: clssgmAllocProc: (0x2327a60) allocated [ CSSD]2013-05-23 16:29:48.949 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:29:48.949 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327a60 - 1,2,3 [ CSSD]2013-05-23 16:29:48.950 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9270) proc(0x2327a60) pid(16816) version 11:1:1:4 [ CSSD]2013-05-23 16:29:48.955 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2327a60), client(1/0x2328220) [ CSSD]2013-05-23 16:29:48.955 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2328220) [ CSSD]2013-05-23 16:29:48.955 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2328220 with con 0x2327d10, requested num -1 [ CSSD]2013-05-23 16:29:48.955 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:48.955 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1547 (0x7f431bfef590) [ CSSD]2013-05-23 16:29:48.956 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef590 (RPC#1547) tag(60b002c) sent to node 1 [ CSSD]2013-05-23 16:29:48.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(060b002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:48.956 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014609, gin 1, member 0 [ CSSD]2013-05-23 16:29:48.956 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328390/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:29:48.957 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:29:48.957 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:29:48.957 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef590 (RPC#1547) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:48.957 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfef590, ret 0, client 0x2328220 member 0x2327940 [ CSSD]2013-05-23 16:29:48.957 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2328220 [ CSSD]2013-05-23 16:29:48.957 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1547 [ CSSD]2013-05-23 16:29:48.963 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2328220) [ CSSD]2013-05-23 16:29:48.963 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2328220), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:48.963 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:29:48.963 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1548 (0x7f431bfef610) [ CSSD]2013-05-23 16:29:48.963 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef610 (RPC#1548) tag(60c002c) sent to node 1 [ CSSD]2013-05-23 16:29:48.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(060c002c) incarnation(174631051) [ CSSD]2013-05-23 16:29:48.963 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:29:48.963 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef610 (RPC#1548) state 6, flags 0x100 [ CSSD]2013-05-23 16:29:48.963 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef610, ret 0, client 0x2328220 member 0x2327940 [ CSSD]2013-05-23 16:29:48.964 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1548 [ CSSD]2013-05-23 16:29:48.964 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:29:49.166 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327a60 [ CSSD]2013-05-23 16:29:49.166 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327a60) con(0x22e9270) skgpid ospid 16816 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:29:49.826 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:49.826 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:50.627 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:50.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:50.809 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:52.346 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:29:52.346 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1549 (0x7f431bfef690) [ CSSD]2013-05-23 16:29:52.347 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef690 (RPC#1549) tag(60d002c) sent to node 1 [ CSSD]2013-05-23 16:29:52.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:52.347 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1549 (0x7f431bfef690) [ CSSD]2013-05-23 16:29:52.347 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef690 (RPC#1549) state 6, flags 0x202 [ CSSD]2013-05-23 16:29:52.347 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef690, ret 0, client (nil) [ CSSD]2013-05-23 16:29:52.347 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1549 [ CSSD]2013-05-23 16:29:52.619 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:52.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:52.834 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:53.829 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:53.829 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:29:54.625 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:54.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:54.867 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:56.631 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:56.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:56.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:58.642 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:29:58.832 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:29:58.832 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:29:58.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:58.899 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:29:59.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:59.237 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014610, gin 1, member 0 [ CSSD]2013-05-23 16:29:59.237 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:29:59.237 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:29:59.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:29:59.242 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:29:59.242 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:30:00.648 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:00.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:00.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:02.375 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:30:02.376 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1550 (0x7f431bfef710) [ CSSD]2013-05-23 16:30:02.376 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef710 (RPC#1550) tag(60e002c) sent to node 1 [ CSSD]2013-05-23 16:30:02.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:02.376 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1550 (0x7f431bfef710) [ CSSD]2013-05-23 16:30:02.376 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef710 (RPC#1550) state 6, flags 0x202 [ CSSD]2013-05-23 16:30:02.376 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef710, ret 0, client (nil) [ CSSD]2013-05-23 16:30:02.377 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1550 [ CSSD]2013-05-23 16:30:02.659 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:02.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:02.916 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:03.835 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:03.835 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:04.197 [1191196992] >TRACE: clssgmAllocProc: (0x2327a60) allocated [ CSSD]2013-05-23 16:30:04.198 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:30:04.199 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327a60 - 1,2,3 [ CSSD]2013-05-23 16:30:04.199 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9800) proc(0x2327a60) pid(17143) version 11:1:1:4 [ CSSD]2013-05-23 16:30:04.204 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2327a60), client(1/0x2328220) [ CSSD]2013-05-23 16:30:04.204 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2328220) [ CSSD]2013-05-23 16:30:04.204 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2328220 with con 0x2327d10, requested num -1 [ CSSD]2013-05-23 16:30:04.204 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:04.204 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1551 (0x7f431bfef790) [ CSSD]2013-05-23 16:30:04.205 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef790 (RPC#1551) tag(60f002c) sent to node 1 [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(060f002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014611, gin 1, member 0 [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328390/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef790 (RPC#1551) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfef790, ret 0, client 0x2328220 member 0x2327940 [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2328220 [ CSSD]2013-05-23 16:30:04.205 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1551 [ CSSD]2013-05-23 16:30:04.212 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2328220) [ CSSD]2013-05-23 16:30:04.212 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2328220), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:04.212 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:04.212 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1552 (0x7f431bfef810) [ CSSD]2013-05-23 16:30:04.212 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef810 (RPC#1552) tag(610002c) sent to node 1 [ CSSD]2013-05-23 16:30:04.212 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0610002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:04.213 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:04.213 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef810 (RPC#1552) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:04.213 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef810, ret 0, client 0x2328220 member 0x2327940 [ CSSD]2013-05-23 16:30:04.213 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1552 [ CSSD]2013-05-23 16:30:04.213 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:30:04.415 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327a60 [ CSSD]2013-05-23 16:30:04.415 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327a60) con(0x22e9800) skgpid ospid 17143 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:04.665 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:04.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:04.968 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:06.671 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:06.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:06.992 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:08.022 [1191196992] >TRACE: clssgmAllocProc: (0x2327a60) allocated [ CSSD]2013-05-23 16:30:08.023 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:30:08.023 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327a60 - 1,2,3 [ CSSD]2013-05-23 16:30:08.023 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9270) proc(0x2327a60) pid(17218) version 11:1:1:4 [ CSSD]2013-05-23 16:30:08.613 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327a60 [ CSSD]2013-05-23 16:30:08.613 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327a60) con(0x22e9270) skgpid ospid 17218 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:08.692 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:08.838 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:08.838 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:09.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:09.005 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:10.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:11.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:11.011 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:12.404 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:30:12.405 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1553 (0x7f431bfef890) [ CSSD]2013-05-23 16:30:12.405 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef890 (RPC#1553) tag(611002c) sent to node 1 [ CSSD]2013-05-23 16:30:12.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:12.406 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1553 (0x7f431bfef890) [ CSSD]2013-05-23 16:30:12.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef890 (RPC#1553) state 6, flags 0x202 [ CSSD]2013-05-23 16:30:12.406 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfef890, ret 0, client (nil) [ CSSD]2013-05-23 16:30:12.406 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1553 [ CSSD]2013-05-23 16:30:12.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:13.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:13.016 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:13.841 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:13.841 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:14.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:14.486 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014612, gin 1, member 0 [ CSSD]2013-05-23 16:30:14.486 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:30:14.486 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:30:14.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:14.492 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:30:14.492 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:30:14.736 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:15.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:15.069 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:16.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:17.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:17.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:17.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:17.915 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014613, gin 1, member 0 [ CSSD]2013-05-23 16:30:17.915 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:30:17.915 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:30:17.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:17.921 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:30:17.921 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:30:18.752 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:18.844 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:18.844 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:19.086 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:19.086 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:19.263 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 16:30:19.263 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f7ce0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:30:19.263 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f7ce0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:30:19.263 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:30:19.263 [1191196992] >TRACE: clssgmDeadProc: proc 0x22eec70 [ CSSD]2013-05-23 16:30:19.263 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22eec70) con(0x230f280) skgpid 12105 ospid 12105 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:19.446 [1191196992] >TRACE: clssgmAllocProc: (0x22f8740) allocated [ CSSD]2013-05-23 16:30:19.447 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:30:19.447 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8740 - 1,2,3 [ CSSD]2013-05-23 16:30:19.447 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22f8740) pid(17401) version 11:1:1:4 [ CSSD]2013-05-23 16:30:19.453 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f8740), client(1/0x22eeec0) [ CSSD]2013-05-23 16:30:19.453 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eeec0) [ CSSD]2013-05-23 16:30:19.453 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eeec0 with con 0x22f7cd0, requested num -1 [ CSSD]2013-05-23 16:30:19.453 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:19.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1554 (0x7f431bfef910) [ CSSD]2013-05-23 16:30:19.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef910 (RPC#1554) tag(612002c) sent to node 1 [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0612002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014614, gin 1, member 0 [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9550/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef910 (RPC#1554) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfef910, ret 0, client 0x22eeec0 member 0x22ff760 [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eeec0 [ CSSD]2013-05-23 16:30:19.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1554 [ CSSD]2013-05-23 16:30:19.460 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eeec0) [ CSSD]2013-05-23 16:30:19.460 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eeec0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:19.460 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ff760) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:19.460 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1555 (0x7f431bfef990) [ CSSD]2013-05-23 16:30:19.460 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfef990 (RPC#1555) tag(613002c) sent to node 1 [ CSSD]2013-05-23 16:30:19.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0613002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:19.461 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:19.461 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfef990 (RPC#1555) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:19.461 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfef990, ret 0, client 0x22eeec0 member 0x22ff760 [ CSSD]2013-05-23 16:30:19.461 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1555 [ CSSD]2013-05-23 16:30:19.461 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ff760) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:30:19.663 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f8740 [ CSSD]2013-05-23 16:30:19.663 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f8740) con(0x230f280) skgpid ospid 17401 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:20.757 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:21.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:21.092 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:21.704 [1191196992] >TRACE: clssgmAllocProc: (0x22f8740) allocated [ CSSD]2013-05-23 16:30:21.705 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:30:21.706 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8740 - 1,2,3 [ CSSD]2013-05-23 16:30:21.706 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22f8740) pid(17454) version 11:1:1:4 [ CSSD]2013-05-23 16:30:21.713 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f8740), client(1/0x22eeec0) [ CSSD]2013-05-23 16:30:21.713 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eeec0) [ CSSD]2013-05-23 16:30:21.713 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eeec0 with con 0x22f7cd0, requested num -1 [ CSSD]2013-05-23 16:30:21.713 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:21.713 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1556 (0x7f431bfefa10) [ CSSD]2013-05-23 16:30:21.713 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefa10 (RPC#1556) tag(614002c) sent to node 1 [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0614002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014615, gin 1, member 0 [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9550/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefa10 (RPC#1556) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfefa10, ret 0, client 0x22eeec0 member 0x22ff760 [ CSSD]2013-05-23 16:30:21.714 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eeec0 [ CSSD]2013-05-23 16:30:21.715 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1556 [ CSSD]2013-05-23 16:30:21.721 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eeec0) [ CSSD]2013-05-23 16:30:21.721 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eeec0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:21.721 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ff760) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:21.721 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1557 (0x7f431bfefa90) [ CSSD]2013-05-23 16:30:21.721 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefa90 (RPC#1557) tag(615002c) sent to node 1 [ CSSD]2013-05-23 16:30:21.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0615002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:21.721 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:21.721 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefa90 (RPC#1557) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:21.721 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfefa90, ret 0, client 0x22eeec0 member 0x22ff760 [ CSSD]2013-05-23 16:30:21.721 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1557 [ CSSD]2013-05-23 16:30:21.721 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ff760) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:30:21.924 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f8740 [ CSSD]2013-05-23 16:30:21.924 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f8740) con(0x230f280) skgpid ospid 17454 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:22.428 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:30:22.429 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1558 (0x7f431bfefb10) [ CSSD]2013-05-23 16:30:22.429 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefb10 (RPC#1558) tag(616002c) sent to node 1 [ CSSD]2013-05-23 16:30:22.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:22.429 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1558 (0x7f431bfefb10) [ CSSD]2013-05-23 16:30:22.429 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefb10 (RPC#1558) state 6, flags 0x202 [ CSSD]2013-05-23 16:30:22.429 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfefb10, ret 0, client (nil) [ CSSD]2013-05-23 16:30:22.429 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1558 [ CSSD]2013-05-23 16:30:22.770 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:23.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:23.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:23.847 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:23.847 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:24.776 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:25.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:25.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:26.782 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:27.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:27.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:28.832 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:28.850 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:28.850 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:29.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:29.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:29.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:29.734 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014616, gin 1, member 0 [ CSSD]2013-05-23 16:30:29.734 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:30:29.734 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:30:29.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:29.740 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:30:29.740 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:30:30.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:31.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:31.192 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:32.454 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:30:32.454 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1559 (0x7f431bfefb90) [ CSSD]2013-05-23 16:30:32.454 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefb90 (RPC#1559) tag(617002c) sent to node 1 [ CSSD]2013-05-23 16:30:32.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:32.455 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1559 (0x7f431bfefb90) [ CSSD]2013-05-23 16:30:32.455 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefb90 (RPC#1559) state 6, flags 0x202 [ CSSD]2013-05-23 16:30:32.455 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfefb90, ret 0, client (nil) [ CSSD]2013-05-23 16:30:32.455 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1559 [ CSSD]2013-05-23 16:30:32.849 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:33.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:33.197 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:33.852 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:33.853 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:34.695 [1191196992] >TRACE: clssgmAllocProc: (0x22f8740) allocated [ CSSD]2013-05-23 16:30:34.696 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:30:34.697 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8740 - 1,2,3 [ CSSD]2013-05-23 16:30:34.697 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22f8740) pid(17547) version 11:1:1:4 [ CSSD]2013-05-23 16:30:34.713 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f8740), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:30:34.713 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:30:34.713 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:30:34.713 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:34.713 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1560 (0x7f431bfefc10) [ CSSD]2013-05-23 16:30:34.713 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefc10 (RPC#1560) tag(618002c) sent to node 1 [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0618002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014617, gin 1, member 0 [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefc10 (RPC#1560) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfefc10, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:30:34.714 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1560 [ CSSD]2013-05-23 16:30:34.720 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:30:34.720 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:34.720 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:34.720 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1561 (0x7f431bfefc90) [ CSSD]2013-05-23 16:30:34.720 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefc90 (RPC#1561) tag(619002c) sent to node 1 [ CSSD]2013-05-23 16:30:34.720 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0619002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:34.720 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:34.720 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefc90 (RPC#1561) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:34.720 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfefc90, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 16:30:34.721 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1561 [ CSSD]2013-05-23 16:30:34.721 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:30:34.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:34.923 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f8740 [ CSSD]2013-05-23 16:30:34.923 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f8740) con(0x230f280) skgpid ospid 17547 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:35.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:35.270 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:36.860 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:37.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:37.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:37.855 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:37.855 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:30:38.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:39.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:39.288 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:40.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:41.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:41.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:42.479 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:30:42.479 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1562 (0x7f431bfefd10) [ CSSD]2013-05-23 16:30:42.480 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefd10 (RPC#1562) tag(61a002c) sent to node 1 [ CSSD]2013-05-23 16:30:42.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:42.480 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1562 (0x7f431bfefd10) [ CSSD]2013-05-23 16:30:42.480 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefd10 (RPC#1562) state 6, flags 0x202 [ CSSD]2013-05-23 16:30:42.480 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfefd10, ret 0, client (nil) [ CSSD]2013-05-23 16:30:42.480 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1562 [ CSSD]2013-05-23 16:30:42.858 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:42.858 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:42.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:43.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:43.339 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:44.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:44.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:44.993 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014618, gin 1, member 0 [ CSSD]2013-05-23 16:30:44.993 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:30:44.993 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:30:45.000 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:45.000 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:30:45.000 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a190) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:30:45.370 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:45.370 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:46.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:47.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:47.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:47.861 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:47.862 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:48.953 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:49.378 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:49.378 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:49.965 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 16:30:49.966 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:30:49.967 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 16:30:49.967 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c12f520) pid(17726) version 11:1:1:4 [ CSSD]2013-05-23 16:30:49.983 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12f520), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:30:49.983 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:30:49.983 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:30:49.983 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:49.983 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1563 (0x7f431bfefd90) [ CSSD]2013-05-23 16:30:49.983 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefd90 (RPC#1563) tag(61b002c) sent to node 1 [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(061b002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014619, gin 1, member 0 [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefd90 (RPC#1563) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfefd90, ret 0, client 0x7f431c0e5910 member 0x7f431c095660 [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:30:49.984 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1563 [ CSSD]2013-05-23 16:30:49.991 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:30:49.991 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:49.991 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:49.991 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1564 (0x7f431bfefe10) [ CSSD]2013-05-23 16:30:49.991 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefe10 (RPC#1564) tag(61c002c) sent to node 1 [ CSSD]2013-05-23 16:30:49.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(061c002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:49.992 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:49.992 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefe10 (RPC#1564) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:49.992 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfefe10, ret 0, client 0x7f431c0e5910 member 0x7f431c095660 [ CSSD]2013-05-23 16:30:49.992 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1564 [ CSSD]2013-05-23 16:30:49.992 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:30:50.194 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 16:30:50.194 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c0840c0) skgpid ospid 17726 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:50.959 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:51.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:51.423 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:52.511 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:30:52.511 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1565 (0x7f431bfefe90) [ CSSD]2013-05-23 16:30:52.511 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfefe90 (RPC#1565) tag(61d002c) sent to node 1 [ CSSD]2013-05-23 16:30:52.512 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:52.512 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1565 (0x7f431bfefe90) [ CSSD]2013-05-23 16:30:52.512 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfefe90 (RPC#1565) state 6, flags 0x202 [ CSSD]2013-05-23 16:30:52.512 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfefe90, ret 0, client (nil) [ CSSD]2013-05-23 16:30:52.512 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1565 [ CSSD]2013-05-23 16:30:52.865 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:52.865 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:53.051 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:53.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:53.429 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:55.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:55.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:55.481 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:56.985 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 16:30:56.986 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:30:56.987 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 16:30:56.987 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c048500) pid(17819) version 11:1:1:4 [ CSSD]2013-05-23 16:30:56.993 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c048500), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:30:56.993 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:30:56.993 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:30:56.993 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:56.993 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1566 (0x7f431bfeff10) [ CSSD]2013-05-23 16:30:56.993 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeff10 (RPC#1566) tag(61e002c) sent to node 1 [ CSSD]2013-05-23 16:30:56.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(061e002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:56.993 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014620, gin 1, member 0 [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeff10 (RPC#1566) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfeff10, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:30:56.994 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1566 [ CSSD]2013-05-23 16:30:57.001 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:30:57.001 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:57.001 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:30:57.001 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1567 (0x7f431bfeff90) [ CSSD]2013-05-23 16:30:57.001 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfeff90 (RPC#1567) tag(61f002c) sent to node 1 [ CSSD]2013-05-23 16:30:57.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(061f002c) incarnation(174631051) [ CSSD]2013-05-23 16:30:57.001 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:30:57.001 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfeff90 (RPC#1567) state 6, flags 0x100 [ CSSD]2013-05-23 16:30:57.001 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfeff90, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 16:30:57.001 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1567 [ CSSD]2013-05-23 16:30:57.001 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:30:57.102 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:57.204 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 16:30:57.204 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c12f0c0) skgpid ospid 17819 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:30:57.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:57.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:30:57.868 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:30:57.868 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:30:59.195 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:30:59.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:30:59.499 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:00.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:00.242 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014621, gin 1, member 0 [ CSSD]2013-05-23 16:31:00.242 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:31:00.242 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:31:00.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:00.248 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:31:00.248 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:31:01.201 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:01.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:01.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:02.534 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:31:02.535 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1568 (0x7f431bff0010) [ CSSD]2013-05-23 16:31:02.535 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0010 (RPC#1568) tag(620002c) sent to node 1 [ CSSD]2013-05-23 16:31:02.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:02.535 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1568 (0x7f431bff0010) [ CSSD]2013-05-23 16:31:02.535 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0010 (RPC#1568) state 6, flags 0x202 [ CSSD]2013-05-23 16:31:02.535 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0010, ret 0, client (nil) [ CSSD]2013-05-23 16:31:02.536 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1568 [ CSSD]2013-05-23 16:31:02.872 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:02.872 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:03.232 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:03.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:03.510 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:05.235 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 16:31:05.236 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:31:05.237 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 16:31:05.237 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c12f520) pid(17917) version 11:1:1:4 [ CSSD]2013-05-23 16:31:05.237 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:05.242 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12f520), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:31:05.242 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:31:05.242 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 16:31:05.242 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:05.242 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1569 (0x7f431bff0090) [ CSSD]2013-05-23 16:31:05.243 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0090 (RPC#1569) tag(621002c) sent to node 1 [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0621002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014622, gin 1, member 0 [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0090 (RPC#1569) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0090, ret 0, client 0x7f431c0e5910 member 0x7f431c13fa20 [ CSSD]2013-05-23 16:31:05.243 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:31:05.244 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1569 [ CSSD]2013-05-23 16:31:05.249 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:31:05.249 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:05.249 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13fa20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:05.249 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1570 (0x7f431bff0110) [ CSSD]2013-05-23 16:31:05.249 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0110 (RPC#1570) tag(622002c) sent to node 1 [ CSSD]2013-05-23 16:31:05.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0622002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:05.250 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:05.250 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0110 (RPC#1570) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:05.250 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0110, ret 0, client 0x7f431c0e5910 member 0x7f431c13fa20 [ CSSD]2013-05-23 16:31:05.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1570 [ CSSD]2013-05-23 16:31:05.250 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13fa20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:31:05.453 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 16:31:05.453 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c0e6720) skgpid ospid 17917 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:05.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:05.542 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:07.243 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:07.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:07.568 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:07.875 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:07.875 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:09.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:09.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:09.579 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:11.261 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:11.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:11.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:12.564 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:31:12.565 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1571 (0x7f431bff0190) [ CSSD]2013-05-23 16:31:12.565 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0190 (RPC#1571) tag(623002c) sent to node 1 [ CSSD]2013-05-23 16:31:12.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:12.565 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1571 (0x7f431bff0190) [ CSSD]2013-05-23 16:31:12.565 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0190 (RPC#1571) state 6, flags 0x202 [ CSSD]2013-05-23 16:31:12.565 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0190, ret 0, client (nil) [ CSSD]2013-05-23 16:31:12.566 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1571 [ CSSD]2013-05-23 16:31:12.878 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:12.878 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:13.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:13.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:13.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:15.317 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:15.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:15.492 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014623, gin 1, member 0 [ CSSD]2013-05-23 16:31:15.492 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:31:15.492 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:31:15.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:15.499 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:31:15.499 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:31:15.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:15.601 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:16.881 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:16.881 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:31:17.323 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:17.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:17.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:17.899 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 16:31:17.900 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:31:17.901 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 16:31:17.901 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c048500) pid(18049) version 11:1:1:4 [ CSSD]2013-05-23 16:31:17.918 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c048500), client(1/0x7f431c10a560) [ CSSD]2013-05-23 16:31:17.918 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:31:17.918 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c10a560 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:31:17.918 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:31:17.918 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1572 (0x7f431bff0210) [ CSSD]2013-05-23 16:31:17.918 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0210 (RPC#1572) tag(624002c) sent to node 1 [ CSSD]2013-05-23 16:31:17.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0624002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:17.918 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014624, gin 1, member 0 [ CSSD]2013-05-23 16:31:17.918 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:31:17.919 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:31:17.919 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:31:17.919 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0210 (RPC#1572) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:17.919 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0210, ret 0, client 0x7f431c10a560 member 0x7f431c095660 [ CSSD]2013-05-23 16:31:17.919 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:31:17.919 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1572 [ CSSD]2013-05-23 16:31:17.925 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 16:31:17.925 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c10a560), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:31:17.925 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:31:17.925 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1573 (0x7f431bff0290) [ CSSD]2013-05-23 16:31:17.925 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0290 (RPC#1573) tag(625002c) sent to node 1 [ CSSD]2013-05-23 16:31:17.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0625002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:17.926 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:31:17.926 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0290 (RPC#1573) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:17.926 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0290, ret 0, client 0x7f431c10a560 member 0x7f431c095660 [ CSSD]2013-05-23 16:31:17.926 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1573 [ CSSD]2013-05-23 16:31:17.926 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:31:18.038 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1215d0) allocated [ CSSD]2013-05-23 16:31:18.039 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:31:18.040 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1215d0 - 1,2,3 [ CSSD]2013-05-23 16:31:18.040 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1215d0) pid(18054) version 11:1:1:4 [ CSSD]2013-05-23 16:31:18.056 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:31:18.057 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:31:18.060 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1215d0 [ CSSD]2013-05-23 16:31:18.060 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1215d0) con(0x7f431c108e60) skgpid ospid 18054 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:18.128 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 16:31:18.128 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c0840c0) skgpid ospid 18049 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:18.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:18.174 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014625, gin 1, member 0 [ CSSD]2013-05-23 16:31:18.174 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:31:18.174 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:31:18.180 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:18.180 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:31:18.180 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:31:19.335 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:19.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:19.699 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:20.489 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 16:31:20.490 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:31:20.491 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 16:31:20.491 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c048500) pid(18073) version 11:1:1:4 [ CSSD]2013-05-23 16:31:20.497 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c048500), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:31:20.497 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:31:20.497 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:31:20.497 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:20.497 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1574 (0x7f431bff0310) [ CSSD]2013-05-23 16:31:20.497 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0310 (RPC#1574) tag(626002c) sent to node 1 [ CSSD]2013-05-23 16:31:20.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0626002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:20.497 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014626, gin 1, member 0 [ CSSD]2013-05-23 16:31:20.497 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:31:20.498 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:31:20.498 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:31:20.498 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0310 (RPC#1574) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:20.498 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0310, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 16:31:20.498 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:31:20.498 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1574 [ CSSD]2013-05-23 16:31:20.503 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:31:20.503 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:20.503 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:20.503 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1575 (0x7f431bff0390) [ CSSD]2013-05-23 16:31:20.503 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0390 (RPC#1575) tag(627002c) sent to node 1 [ CSSD]2013-05-23 16:31:20.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0627002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:20.503 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:20.504 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0390 (RPC#1575) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:20.504 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0390, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 16:31:20.504 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1575 [ CSSD]2013-05-23 16:31:20.504 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:31:20.706 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 16:31:20.706 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c12f0c0) skgpid ospid 18073 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:21.341 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:21.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:21.705 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:21.884 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:21.884 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:21.967 [1191196992] >TRACE: clssgmAllocProc: (0x22f8740) allocated [ CSSD]2013-05-23 16:31:21.968 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:31:21.969 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8740 - 1,2,3 [ CSSD]2013-05-23 16:31:21.969 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22f8740) pid(18131) version 11:1:1:4 [ CSSD]2013-05-23 16:31:21.975 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f8740), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:31:21.975 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:31:21.975 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:31:21.975 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:21.975 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1576 (0x7f431bff0410) [ CSSD]2013-05-23 16:31:21.975 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0410 (RPC#1576) tag(628002c) sent to node 1 [ CSSD]2013-05-23 16:31:21.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0628002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:21.975 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014627, gin 1, member 0 [ CSSD]2013-05-23 16:31:21.975 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:31:21.976 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:31:21.976 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:31:21.976 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0410 (RPC#1576) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:21.976 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0410, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 16:31:21.976 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:31:21.976 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1576 [ CSSD]2013-05-23 16:31:21.982 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:31:21.982 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:21.982 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:21.982 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1577 (0x7f431bff0490) [ CSSD]2013-05-23 16:31:21.982 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0490 (RPC#1577) tag(629002c) sent to node 1 [ CSSD]2013-05-23 16:31:21.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0629002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:21.983 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:21.983 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0490 (RPC#1577) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:21.983 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0490, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 16:31:21.983 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1577 [ CSSD]2013-05-23 16:31:21.983 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:31:22.185 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f8740 [ CSSD]2013-05-23 16:31:22.185 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f8740) con(0x230f280) skgpid ospid 18131 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:22.591 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:31:22.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1578 (0x7f431bff0510) [ CSSD]2013-05-23 16:31:22.592 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0510 (RPC#1578) tag(62a002c) sent to node 1 [ CSSD]2013-05-23 16:31:22.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:22.592 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1578 (0x7f431bff0510) [ CSSD]2013-05-23 16:31:22.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0510 (RPC#1578) state 6, flags 0x202 [ CSSD]2013-05-23 16:31:22.592 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0510, ret 0, client (nil) [ CSSD]2013-05-23 16:31:22.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1578 [ CSSD]2013-05-23 16:31:23.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:23.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:23.711 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:23.873 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:31:23.873 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:31:23.873 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:31:23.873 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:31:23.876 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2317dd0) [ CSSD]2013-05-23 16:31:23.876 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2317dd0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:31:23.876 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2317dd0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:31:23.876 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:31:23.881 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f4c00) [ CSSD]2013-05-23 16:31:23.881 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f4c00), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:31:23.881 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f4c00), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:31:23.881 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:31:23.882 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2ed0 [ CSSD]2013-05-23 16:31:23.882 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2ed0) con(0x2317960) skgpid 12800 ospid 12800 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:23.887 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:31:23.887 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:31:23.887 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:31:23.887 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:31:23.887 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 16:31:23.887 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c0e5200) skgpid 31983 ospid 31983 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:25.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:25.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:25.722 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:26.887 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:26.887 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:27.354 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:27.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:27.729 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:29.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:29.780 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:29.780 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:30.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:30.743 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014628, gin 1, member 0 [ CSSD]2013-05-23 16:31:30.743 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:31:30.743 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:31:30.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:30.749 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:31:30.749 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dff50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:31:31.371 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:31.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:31.785 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:31.890 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:31.891 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:32.614 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:31:32.614 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1579 (0x7f431bff0590) [ CSSD]2013-05-23 16:31:32.614 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0590 (RPC#1579) tag(62b002c) sent to node 1 [ CSSD]2013-05-23 16:31:32.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:32.615 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1579 (0x7f431bff0590) [ CSSD]2013-05-23 16:31:32.615 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0590 (RPC#1579) state 6, flags 0x202 [ CSSD]2013-05-23 16:31:32.615 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0590, ret 0, client (nil) [ CSSD]2013-05-23 16:31:32.615 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1579 [ CSSD]2013-05-23 16:31:33.463 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:33.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:33.790 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:35.469 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:35.736 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 16:31:35.737 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:31:35.737 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 16:31:35.737 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c10a490) pid(18252) version 11:1:1:4 [ CSSD]2013-05-23 16:31:35.743 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c10a490), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:31:35.743 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:31:35.743 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:31:35.743 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:35.743 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1580 (0x7f431bff0610) [ CSSD]2013-05-23 16:31:35.744 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0610 (RPC#1580) tag(62c002c) sent to node 1 [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(062c002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014629, gin 1, member 0 [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0610 (RPC#1580) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0610, ret 0, client 0x7f431c0e6e30 member 0x7f431c109eb0 [ CSSD]2013-05-23 16:31:35.744 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:31:35.745 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1580 [ CSSD]2013-05-23 16:31:35.751 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:31:35.751 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:35.751 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:35.751 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1581 (0x7f431bff0690) [ CSSD]2013-05-23 16:31:35.751 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0690 (RPC#1581) tag(62d002c) sent to node 1 [ CSSD]2013-05-23 16:31:35.751 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(062d002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:35.751 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:35.751 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0690 (RPC#1581) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:35.751 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0690, ret 0, client 0x7f431c0e6e30 member 0x7f431c109eb0 [ CSSD]2013-05-23 16:31:35.752 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1581 [ CSSD]2013-05-23 16:31:35.752 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:31:35.803 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:35.803 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:35.954 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 16:31:35.954 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e6720) skgpid ospid 18252 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:36.894 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:36.894 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:37.475 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:37.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:37.808 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:39.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:39.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:39.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:41.490 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:41.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:41.825 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:41.897 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:41.897 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:42.637 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:31:42.637 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1582 (0x7f431bff0710) [ CSSD]2013-05-23 16:31:42.637 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0710 (RPC#1582) tag(62e002c) sent to node 1 [ CSSD]2013-05-23 16:31:42.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:42.638 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1582 (0x7f431bff0710) [ CSSD]2013-05-23 16:31:42.638 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0710 (RPC#1582) state 6, flags 0x202 [ CSSD]2013-05-23 16:31:42.638 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0710, ret 0, client (nil) [ CSSD]2013-05-23 16:31:42.638 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1582 [ CSSD]2013-05-23 16:31:43.543 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:43.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:43.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:45.548 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:45.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:45.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:46.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:46.003 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014630, gin 1, member 0 [ CSSD]2013-05-23 16:31:46.003 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:31:46.003 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:31:46.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:46.009 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:31:46.009 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dff50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:31:46.900 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:46.900 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:47.561 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:47.907 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:47.907 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:49.567 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:50.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:50.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:50.985 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c109eb0) allocated [ CSSD]2013-05-23 16:31:50.986 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:31:50.986 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c109eb0 - 1,2,3 [ CSSD]2013-05-23 16:31:50.986 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c109eb0) pid(18406) version 11:1:1:4 [ CSSD]2013-05-23 16:31:50.992 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c109eb0), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:31:50.992 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:31:50.992 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 16:31:50.992 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:50.992 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1583 (0x7f431bff0790) [ CSSD]2013-05-23 16:31:50.992 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0790 (RPC#1583) tag(62f002c) sent to node 1 [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(062f002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014631, gin 1, member 0 [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0790 (RPC#1583) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0790, ret 0, client 0x7f431c135120 member 0x7f431c0e5840 [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:31:50.993 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1583 [ CSSD]2013-05-23 16:31:50.999 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:31:50.999 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:50.999 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:31:50.999 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1584 (0x7f431bff0810) [ CSSD]2013-05-23 16:31:50.999 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0810 (RPC#1584) tag(630002c) sent to node 1 [ CSSD]2013-05-23 16:31:50.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0630002c) incarnation(174631051) [ CSSD]2013-05-23 16:31:50.999 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:31:50.999 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0810 (RPC#1584) state 6, flags 0x100 [ CSSD]2013-05-23 16:31:50.999 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0810, ret 0, client 0x7f431c135120 member 0x7f431c0e5840 [ CSSD]2013-05-23 16:31:51.000 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1584 [ CSSD]2013-05-23 16:31:51.000 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:31:51.202 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c109eb0 [ CSSD]2013-05-23 16:31:51.202 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c109eb0) con(0x7f431c11ab50) skgpid ospid 18406 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:31:51.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:51.903 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:51.903 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:52.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:52.007 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:52.662 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:31:52.662 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1585 (0x7f431bff0890) [ CSSD]2013-05-23 16:31:52.662 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0890 (RPC#1585) tag(631002c) sent to node 1 [ CSSD]2013-05-23 16:31:52.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:52.663 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1585 (0x7f431bff0890) [ CSSD]2013-05-23 16:31:52.663 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0890 (RPC#1585) state 6, flags 0x202 [ CSSD]2013-05-23 16:31:52.663 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0890, ret 0, client (nil) [ CSSD]2013-05-23 16:31:52.663 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1585 [ CSSD]2013-05-23 16:31:53.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:54.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:54.019 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:55.589 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:56.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:56.024 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:56.905 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:31:56.906 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:31:57.642 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:31:58.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:31:58.030 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:31:59.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:00.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:00.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:00.908 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:00.908 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:32:01.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:01.251 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014632, gin 1, member 0 [ CSSD]2013-05-23 16:32:01.251 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:32:01.251 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:32:01.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:01.258 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:32:01.258 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dff50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:32:01.673 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:02.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:02.087 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:02.688 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:32:02.688 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1586 (0x7f431bff0910) [ CSSD]2013-05-23 16:32:02.688 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0910 (RPC#1586) tag(632002c) sent to node 1 [ CSSD]2013-05-23 16:32:02.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:02.689 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1586 (0x7f431bff0910) [ CSSD]2013-05-23 16:32:02.689 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0910 (RPC#1586) state 6, flags 0x202 [ CSSD]2013-05-23 16:32:02.689 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0910, ret 0, client (nil) [ CSSD]2013-05-23 16:32:02.689 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1586 [ CSSD]2013-05-23 16:32:03.685 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:04.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:04.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:05.691 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:05.911 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:05.912 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:06.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:06.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:06.244 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e6d60) allocated [ CSSD]2013-05-23 16:32:06.245 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:32:06.246 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e6d60 - 1,2,3 [ CSSD]2013-05-23 16:32:06.246 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c0e6d60) pid(18552) version 11:1:1:4 [ CSSD]2013-05-23 16:32:06.252 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0e6d60), client(1/0x7f431c108e70) [ CSSD]2013-05-23 16:32:06.252 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 16:32:06.252 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:32:06.252 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:06.252 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1587 (0x7f431bff0990) [ CSSD]2013-05-23 16:32:06.252 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0990 (RPC#1587) tag(633002c) sent to node 1 [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0633002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014633, gin 1, member 0 [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0990 (RPC#1587) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0990, ret 0, client 0x7f431c108e70 member 0x7f431c10a490 [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 16:32:06.253 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1587 [ CSSD]2013-05-23 16:32:06.260 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 16:32:06.260 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:06.260 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:06.260 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1588 (0x7f431bff0a10) [ CSSD]2013-05-23 16:32:06.260 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0a10 (RPC#1588) tag(634002c) sent to node 1 [ CSSD]2013-05-23 16:32:06.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0634002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:06.261 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:06.261 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0a10 (RPC#1588) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:06.261 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0a10, ret 0, client 0x7f431c108e70 member 0x7f431c10a490 [ CSSD]2013-05-23 16:32:06.261 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1588 [ CSSD]2013-05-23 16:32:06.261 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:06.463 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e6d60 [ CSSD]2013-05-23 16:32:06.463 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e6d60) con(0x7f431c12a550) skgpid ospid 18552 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:07.040 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 16:32:07.041 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:32:07.042 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 16:32:07.042 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c10a490) pid(18616) version 11:1:1:4 [ CSSD]2013-05-23 16:32:07.050 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c10a490), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:32:07.050 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:32:07.050 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 16:32:07.050 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:07.050 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1589 (0x7f431bff0a90) [ CSSD]2013-05-23 16:32:07.050 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0a90 (RPC#1589) tag(635002c) sent to node 1 [ CSSD]2013-05-23 16:32:07.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0635002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:07.050 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014634, gin 1, member 0 [ CSSD]2013-05-23 16:32:07.050 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:32:07.051 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:07.051 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:32:07.051 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0a90 (RPC#1589) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:07.051 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0a90, ret 0, client 0x7f431c0e5910 member 0x7f431c109eb0 [ CSSD]2013-05-23 16:32:07.051 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:32:07.051 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1589 [ CSSD]2013-05-23 16:32:07.058 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:32:07.058 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:07.058 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:07.058 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1590 (0x7f431bff0b10) [ CSSD]2013-05-23 16:32:07.058 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0b10 (RPC#1590) tag(636002c) sent to node 1 [ CSSD]2013-05-23 16:32:07.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0636002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:07.059 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:07.059 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0b10 (RPC#1590) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:07.059 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0b10, ret 0, client 0x7f431c0e5910 member 0x7f431c109eb0 [ CSSD]2013-05-23 16:32:07.059 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1590 [ CSSD]2013-05-23 16:32:07.059 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:07.262 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 16:32:07.262 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c11ab50) skgpid ospid 18616 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:07.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:08.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:08.130 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:09.165 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 16:32:09.166 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:32:09.166 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 16:32:09.166 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c108da0) pid(17399/17399) version 11:1:1:4 [ CSSD]2013-05-23 16:32:09.182 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c108da0), client(1/0x230f280) [ CSSD]2013-05-23 16:32:09.182 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x230f280) [ CSSD]2013-05-23 16:32:09.182 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:32:09.182 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:32:09.748 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:10.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:10.152 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:10.857 [1191196992] >TRACE: clssgmAllocProc: (0x22f7c20) allocated [ CSSD]2013-05-23 16:32:10.858 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:32:10.859 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f7c20 - 1,2,3 [ CSSD]2013-05-23 16:32:10.859 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4b70) proc(0x22f7c20) pid(18164/18164) version 11:1:1:4 [ CSSD]2013-05-23 16:32:10.865 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f7c20), client(1/0x2317ea0) [ CSSD]2013-05-23 16:32:10.865 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2317ea0) [ CSSD]2013-05-23 16:32:10.865 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:32:10.865 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:32:10.915 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:10.915 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:11.293 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f7c20), client(2/0x22f1bb0) [ CSSD]2013-05-23 16:32:11.293 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f1bb0) [ CSSD]2013-05-23 16:32:11.293 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:32:11.293 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:32:11.534 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c108da0), client(2/0x2317b60) [ CSSD]2013-05-23 16:32:11.534 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2317b60) [ CSSD]2013-05-23 16:32:11.534 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:32:11.534 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:32:11.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:12.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:12.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:12.712 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:32:12.712 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1591 (0x7f431bff0b90) [ CSSD]2013-05-23 16:32:12.712 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0b90 (RPC#1591) tag(637002c) sent to node 1 [ CSSD]2013-05-23 16:32:12.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:12.712 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1591 (0x7f431bff0b90) [ CSSD]2013-05-23 16:32:12.712 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0b90 (RPC#1591) state 6, flags 0x202 [ CSSD]2013-05-23 16:32:12.712 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0b90, ret 0, client (nil) [ CSSD]2013-05-23 16:32:12.713 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1591 [ CSSD]2013-05-23 16:32:13.764 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:14.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:14.190 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:15.770 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:15.917 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:15.917 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:16.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:16.216 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:16.394 [1191196992] >TRACE: clssgmAllocProc: (0x22f2c20) allocated [ CSSD]2013-05-23 16:32:16.395 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:32:16.396 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2c20 - 1,2,3 [ CSSD]2013-05-23 16:32:16.396 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f84a0) proc(0x22f2c20) pid(18892) version 11:1:1:4 [ CSSD]2013-05-23 16:32:16.501 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:16.501 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014635, gin 1, member 0 [ CSSD]2013-05-23 16:32:16.501 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f2e80/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:32:16.501 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:32:16.507 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:16.507 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:32:16.507 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f2b00) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:32:16.536 [1191196992] >TRACE: clssgmAllocProc: (0x2328130) allocated [ CSSD]2013-05-23 16:32:16.538 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:32:16.538 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2328130 - 1,2,3 [ CSSD]2013-05-23 16:32:16.538 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3030) proc(0x2328130) pid(18902) version 11:1:1:4 [ CSSD]2013-05-23 16:32:16.554 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:32:16.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:32:16.559 [1191196992] >TRACE: clssgmDeadProc: proc 0x2328130 [ CSSD]2013-05-23 16:32:16.559 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2328130) con(0x22f3030) skgpid ospid 18902 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:16.636 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(69/0x22e9e90) [ CSSD]2013-05-23 16:32:16.636 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 69 (0x22e9e90) [ CSSD]2013-05-23 16:32:16.636 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22e9e90 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 16:32:16.636 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:32:16.636 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1592 (0x7f431bff0c10) [ CSSD]2013-05-23 16:32:16.636 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0c10 (RPC#1592) tag(638002c) sent to node 1 [ CSSD]2013-05-23 16:32:16.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0638002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:16.636 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014636, gin 1, member 0 [ CSSD]2013-05-23 16:32:16.636 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x22ea000/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:32:16.637 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:16.637 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:32:16.637 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0c10 (RPC#1592) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:16.637 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0c10, ret 0, client 0x22e9e90 member 0x22f2e80 [ CSSD]2013-05-23 16:32:16.637 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e9e90 [ CSSD]2013-05-23 16:32:16.637 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1592 [ CSSD]2013-05-23 16:32:16.642 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 69 (0x22e9e90) [ CSSD]2013-05-23 16:32:16.642 [1191196992] >TRACE: clssgmExitGrock: client 69 (0x22e9e90), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:32:16.642 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e80) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:32:16.642 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1593 (0x7f431bff0c90) [ CSSD]2013-05-23 16:32:16.643 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0c90 (RPC#1593) tag(639002c) sent to node 1 [ CSSD]2013-05-23 16:32:16.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0639002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:16.643 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:32:16.643 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0c90 (RPC#1593) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:16.643 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0c90, ret 0, client 0x22e9e90 member 0x22f2e80 [ CSSD]2013-05-23 16:32:16.643 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1593 [ CSSD]2013-05-23 16:32:16.643 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22f2e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:16.644 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2c20 [ CSSD]2013-05-23 16:32:16.644 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2c20) con(0x22f84a0) skgpid ospid 18892 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:17.781 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:18.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:18.221 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:18.436 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:18.436 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014637, gin 1, member 0 [ CSSD]2013-05-23 16:32:18.436 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f7f10/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:32:18.436 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:32:18.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:18.443 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:32:18.443 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2317a50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:32:19.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:20.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:20.232 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:20.920 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:20.921 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:21.497 [1191196992] >TRACE: clssgmAllocProc: (0x22f2c20) allocated [ CSSD]2013-05-23 16:32:21.498 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:32:21.498 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2c20 - 1,2,3 [ CSSD]2013-05-23 16:32:21.498 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f84a0) proc(0x22f2c20) pid(18942) version 11:1:1:4 [ CSSD]2013-05-23 16:32:21.504 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f2c20), client(1/0x22e9dd0) [ CSSD]2013-05-23 16:32:21.504 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e9dd0) [ CSSD]2013-05-23 16:32:21.504 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e9dd0 with con 0x22f2e80, requested num -1 [ CSSD]2013-05-23 16:32:21.504 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:21.504 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1594 (0x7f431bff0d10) [ CSSD]2013-05-23 16:32:21.504 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0d10 (RPC#1594) tag(63a002c) sent to node 1 [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(063a002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014638, gin 1, member 0 [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9f40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0d10 (RPC#1594) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0d10, ret 0, client 0x22e9dd0 member 0x22f2b00 [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e9dd0 [ CSSD]2013-05-23 16:32:21.505 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1594 [ CSSD]2013-05-23 16:32:21.511 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e9dd0) [ CSSD]2013-05-23 16:32:21.511 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e9dd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:21.511 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2b00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:21.511 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1595 (0x7f431bff0d90) [ CSSD]2013-05-23 16:32:21.511 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0d90 (RPC#1595) tag(63b002c) sent to node 1 [ CSSD]2013-05-23 16:32:21.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(063b002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:21.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:21.511 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0d90 (RPC#1595) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:21.511 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0d90, ret 0, client 0x22e9dd0 member 0x22f2b00 [ CSSD]2013-05-23 16:32:21.512 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1595 [ CSSD]2013-05-23 16:32:21.512 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2b00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:21.714 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2c20 [ CSSD]2013-05-23 16:32:21.714 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2c20) con(0x22f84a0) skgpid ospid 18942 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:21.853 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:22.225 [1191196992] >TRACE: clssgmAllocProc: (0x22f2cf0) allocated [ CSSD]2013-05-23 16:32:22.226 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:32:22.227 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2cf0 - 1,2,3 [ CSSD]2013-05-23 16:32:22.227 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ee320) proc(0x22f2cf0) pid(18989) version 11:1:1:4 [ CSSD]2013-05-23 16:32:22.233 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f2cf0), client(1/0x22eeed0) [ CSSD]2013-05-23 16:32:22.233 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22eeed0) [ CSSD]2013-05-23 16:32:22.233 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22eeed0 with con 0x22f2f50, requested num -1 [ CSSD]2013-05-23 16:32:22.233 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:22.233 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1596 (0x7f431bff0e10) [ CSSD]2013-05-23 16:32:22.233 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0e10 (RPC#1596) tag(63c002c) sent to node 1 [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(063c002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014639, gin 1, member 0 [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ea020/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0e10 (RPC#1596) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff0e10, ret 0, client 0x22eeed0 member 0x22f2ae0 [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22eeed0 [ CSSD]2013-05-23 16:32:22.234 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1596 [ CSSD]2013-05-23 16:32:22.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:22.237 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:22.240 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22eeed0) [ CSSD]2013-05-23 16:32:22.240 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22eeed0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:22.240 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2ae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:22.240 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1597 (0x7f431bff0e90) [ CSSD]2013-05-23 16:32:22.240 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0e90 (RPC#1597) tag(63d002c) sent to node 1 [ CSSD]2013-05-23 16:32:22.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(063d002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:22.240 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:22.240 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0e90 (RPC#1597) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:22.240 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff0e90, ret 0, client 0x22eeed0 member 0x22f2ae0 [ CSSD]2013-05-23 16:32:22.241 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1597 [ CSSD]2013-05-23 16:32:22.241 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2ae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:22.443 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2cf0 [ CSSD]2013-05-23 16:32:22.443 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2cf0) con(0x22ee320) skgpid ospid 18989 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:22.743 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:32:22.743 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1598 (0x7f431bff0f10) [ CSSD]2013-05-23 16:32:22.744 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0f10 (RPC#1598) tag(63e002c) sent to node 1 [ CSSD]2013-05-23 16:32:22.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:22.744 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1598 (0x7f431bff0f10) [ CSSD]2013-05-23 16:32:22.744 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0f10 (RPC#1598) state 6, flags 0x202 [ CSSD]2013-05-23 16:32:22.744 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0f10, ret 0, client (nil) [ CSSD]2013-05-23 16:32:22.744 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1598 [ CSSD]2013-05-23 16:32:23.866 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:24.290 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:24.290 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:24.923 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:24.923 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:32:25.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:26.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:26.296 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:27.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:28.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:28.301 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:29.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:29.926 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:29.926 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:30.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:30.313 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:31.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:31.756 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014640, gin 1, member 0 [ CSSD]2013-05-23 16:32:31.756 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f7f10/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:32:31.756 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:32:31.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:31.763 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:32:31.763 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2317a50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:32:31.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:32.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:32.339 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:32.771 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:32:32.771 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1599 (0x7f431bff0f90) [ CSSD]2013-05-23 16:32:32.772 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff0f90 (RPC#1599) tag(63f002c) sent to node 1 [ CSSD]2013-05-23 16:32:32.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:32.772 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1599 (0x7f431bff0f90) [ CSSD]2013-05-23 16:32:32.772 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff0f90 (RPC#1599) state 6, flags 0x202 [ CSSD]2013-05-23 16:32:32.772 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff0f90, ret 0, client (nil) [ CSSD]2013-05-23 16:32:32.772 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1599 [ CSSD]2013-05-23 16:32:33.916 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:34.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:34.390 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:34.930 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:34.930 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:35.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:36.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:36.396 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:36.744 [1191196992] >TRACE: clssgmAllocProc: (0x22f2c20) allocated [ CSSD]2013-05-23 16:32:36.745 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:32:36.746 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2c20 - 1,2,3 [ CSSD]2013-05-23 16:32:36.746 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f84a0) proc(0x22f2c20) pid(19117) version 11:1:1:4 [ CSSD]2013-05-23 16:32:36.751 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f2c20), client(1/0x22e9dd0) [ CSSD]2013-05-23 16:32:36.751 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e9dd0) [ CSSD]2013-05-23 16:32:36.751 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e9dd0 with con 0x22f2e80, requested num -1 [ CSSD]2013-05-23 16:32:36.751 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:36.751 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1600 (0x7f431bff1010) [ CSSD]2013-05-23 16:32:36.752 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1010 (RPC#1600) tag(640002c) sent to node 1 [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0640002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014641, gin 1, member 0 [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9f40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1010 (RPC#1600) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1010, ret 0, client 0x22e9dd0 member 0x22f2b00 [ CSSD]2013-05-23 16:32:36.752 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e9dd0 [ CSSD]2013-05-23 16:32:36.753 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1600 [ CSSD]2013-05-23 16:32:36.759 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e9dd0) [ CSSD]2013-05-23 16:32:36.759 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e9dd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:36.759 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2b00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:36.759 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1601 (0x7f431bff1090) [ CSSD]2013-05-23 16:32:36.759 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1090 (RPC#1601) tag(641002c) sent to node 1 [ CSSD]2013-05-23 16:32:36.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0641002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:36.759 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:36.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1090 (RPC#1601) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:36.760 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1090, ret 0, client 0x22e9dd0 member 0x22f2b00 [ CSSD]2013-05-23 16:32:36.760 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1601 [ CSSD]2013-05-23 16:32:36.760 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2b00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:36.962 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2c20 [ CSSD]2013-05-23 16:32:36.962 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2c20) con(0x22f84a0) skgpid ospid 19117 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:37.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:38.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:38.401 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:39.933 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:39.933 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:39.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:40.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:40.413 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:41.966 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:42.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:42.439 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:42.802 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:32:42.802 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1602 (0x7f431bff1110) [ CSSD]2013-05-23 16:32:42.802 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1110 (RPC#1602) tag(642002c) sent to node 1 [ CSSD]2013-05-23 16:32:42.803 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:42.803 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1602 (0x7f431bff1110) [ CSSD]2013-05-23 16:32:42.803 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1110 (RPC#1602) state 6, flags 0x202 [ CSSD]2013-05-23 16:32:42.803 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1110, ret 0, client (nil) [ CSSD]2013-05-23 16:32:42.803 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1602 [ CSSD]2013-05-23 16:32:43.936 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:43.936 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:32:43.967 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:44.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:44.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:45.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:46.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:46.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:47.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:47.006 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014642, gin 1, member 0 [ CSSD]2013-05-23 16:32:47.006 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f7f10/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:32:47.006 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:32:47.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:47.011 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:32:47.011 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2317a50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:32:47.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:48.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:48.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:48.939 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:48.939 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:49.292 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f5100) [ CSSD]2013-05-23 16:32:49.292 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f5100), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:32:49.292 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f5100), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:32:49.292 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:32:49.302 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:32:49.302 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23029c0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:32:49.302 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23029c0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:32:49.302 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:32:49.302 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7550 [ CSSD]2013-05-23 16:32:49.302 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7550) con(0x2302d80) skgpid 13799 ospid 13799 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:49.990 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:50.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:50.514 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:52.003 [1191196992] >TRACE: clssgmAllocProc: (0x2302bf0) allocated [ CSSD]2013-05-23 16:32:52.004 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:32:52.004 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302bf0 - 1,2,3 [ CSSD]2013-05-23 16:32:52.004 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4e40) proc(0x2302bf0) pid(19327) version 11:1:1:4 [ CSSD]2013-05-23 16:32:52.010 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2302bf0), client(1/0x23029c0) [ CSSD]2013-05-23 16:32:52.010 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:32:52.010 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23029c0 with con 0x2302710, requested num -1 [ CSSD]2013-05-23 16:32:52.010 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:52.010 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1603 (0x7f431bff1190) [ CSSD]2013-05-23 16:32:52.010 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1190 (RPC#1603) tag(643002c) sent to node 1 [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0643002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014643, gin 1, member 0 [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1190 (RPC#1603) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1190, ret 0, client 0x23029c0 member 0x22dfd50 [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23029c0 [ CSSD]2013-05-23 16:32:52.011 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1603 [ CSSD]2013-05-23 16:32:52.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:52.017 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:32:52.017 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23029c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:52.017 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22dfd50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:32:52.017 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1604 (0x7f431bff1210) [ CSSD]2013-05-23 16:32:52.017 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1210 (RPC#1604) tag(644002c) sent to node 1 [ CSSD]2013-05-23 16:32:52.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0644002c) incarnation(174631051) [ CSSD]2013-05-23 16:32:52.017 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:32:52.017 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1210 (RPC#1604) state 6, flags 0x100 [ CSSD]2013-05-23 16:32:52.017 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1210, ret 0, client 0x23029c0 member 0x22dfd50 [ CSSD]2013-05-23 16:32:52.017 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1604 [ CSSD]2013-05-23 16:32:52.017 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22dfd50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:32:52.220 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302bf0 [ CSSD]2013-05-23 16:32:52.220 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302bf0) con(0x22f4e40) skgpid ospid 19327 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:32:52.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:52.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:52.827 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:32:52.827 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1605 (0x7f431bff1290) [ CSSD]2013-05-23 16:32:52.827 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1290 (RPC#1605) tag(645002c) sent to node 1 [ CSSD]2013-05-23 16:32:52.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:52.828 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1605 (0x7f431bff1290) [ CSSD]2013-05-23 16:32:52.828 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1290 (RPC#1605) state 6, flags 0x202 [ CSSD]2013-05-23 16:32:52.828 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1290, ret 0, client (nil) [ CSSD]2013-05-23 16:32:52.828 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1605 [ CSSD]2013-05-23 16:32:53.942 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:53.942 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:32:54.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:54.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:54.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:56.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:56.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:56.556 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:58.085 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:32:58.580 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:32:58.580 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:32:58.945 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:32:58.945 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:00.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:00.593 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:00.593 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:02.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:02.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:02.255 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014644, gin 1, member 0 [ CSSD]2013-05-23 16:33:02.255 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:33:02.255 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:33:02.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:02.261 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:33:02.261 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f49f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:33:02.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:02.618 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:02.852 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:33:02.852 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1606 (0x7f431bff1310) [ CSSD]2013-05-23 16:33:02.852 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1310 (RPC#1606) tag(646002c) sent to node 1 [ CSSD]2013-05-23 16:33:02.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:02.853 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1606 (0x7f431bff1310) [ CSSD]2013-05-23 16:33:02.853 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1310 (RPC#1606) state 6, flags 0x202 [ CSSD]2013-05-23 16:33:02.853 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1310, ret 0, client (nil) [ CSSD]2013-05-23 16:33:02.853 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1606 [ CSSD]2013-05-23 16:33:02.947 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:02.948 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:33:04.127 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:04.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:04.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:06.133 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:06.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:06.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:07.259 [1191196992] >TRACE: clssgmAllocProc: (0x22f4dc0) allocated [ CSSD]2013-05-23 16:33:07.259 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:33:07.260 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4dc0 - 1,2,3 [ CSSD]2013-05-23 16:33:07.260 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4ac0) proc(0x22f4dc0) pid(19485) version 11:1:1:4 [ CSSD]2013-05-23 16:33:07.266 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f4dc0), client(1/0x23029c0) [ CSSD]2013-05-23 16:33:07.266 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:33:07.266 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23029c0 with con 0x2302d00, requested num -1 [ CSSD]2013-05-23 16:33:07.266 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:07.266 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1607 (0x7f431bff1390) [ CSSD]2013-05-23 16:33:07.266 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1390 (RPC#1607) tag(647002c) sent to node 1 [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0647002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014645, gin 1, member 0 [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1390 (RPC#1607) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1390, ret 0, client 0x23029c0 member 0x22dfd50 [ CSSD]2013-05-23 16:33:07.266 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23029c0 [ CSSD]2013-05-23 16:33:07.267 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1607 [ CSSD]2013-05-23 16:33:07.272 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:33:07.272 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23029c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:07.272 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22dfd50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:07.272 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1608 (0x7f431bff1410) [ CSSD]2013-05-23 16:33:07.273 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1410 (RPC#1608) tag(648002c) sent to node 1 [ CSSD]2013-05-23 16:33:07.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0648002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:07.273 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:07.273 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1410 (RPC#1608) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:07.273 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1410, ret 0, client 0x23029c0 member 0x22dfd50 [ CSSD]2013-05-23 16:33:07.273 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1608 [ CSSD]2013-05-23 16:33:07.273 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22dfd50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:33:07.475 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4dc0 [ CSSD]2013-05-23 16:33:07.475 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4dc0) con(0x22f4ac0) skgpid ospid 19485 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:33:07.951 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:07.951 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:08.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:08.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:08.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:09.051 [1191196992] >TRACE: clssgmAllocProc: (0x2317a30) allocated [ CSSD]2013-05-23 16:33:09.052 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:33:09.052 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317a30 - 1,2,3 [ CSSD]2013-05-23 16:33:09.053 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4ac0) proc(0x2317a30) pid(19313/19313) version 11:1:1:4 [ CSSD]2013-05-23 16:33:09.058 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2317a30), client(1/0x23029c0) [ CSSD]2013-05-23 16:33:09.058 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:33:09.058 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:33:09.058 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:33:10.171 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:10.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:10.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:12.177 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:12.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:12.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:12.875 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:33:12.876 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1609 (0x7f431bff1490) [ CSSD]2013-05-23 16:33:12.876 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1490 (RPC#1609) tag(649002c) sent to node 1 [ CSSD]2013-05-23 16:33:12.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:12.876 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1609 (0x7f431bff1490) [ CSSD]2013-05-23 16:33:12.876 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1490 (RPC#1609) state 6, flags 0x202 [ CSSD]2013-05-23 16:33:12.876 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1490, ret 0, client (nil) [ CSSD]2013-05-23 16:33:12.876 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1609 [ CSSD]2013-05-23 16:33:12.954 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:12.954 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:14.187 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:14.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:14.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:16.192 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:16.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:16.817 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:17.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:17.504 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014646, gin 1, member 0 [ CSSD]2013-05-23 16:33:17.504 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:33:17.504 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:33:17.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:17.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:33:17.511 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7510) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:33:17.957 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:17.957 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:18.265 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:18.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:18.693 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014647, gin 1, member 0 [ CSSD]2013-05-23 16:33:18.693 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:33:18.693 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:33:18.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:18.699 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:33:18.699 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7420) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:33:18.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:18.823 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:20.271 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:20.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:20.835 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:21.960 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:21.960 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:33:22.295 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:22.490 [1191196992] >TRACE: clssgmAllocProc: (0x22eed10) allocated [ CSSD]2013-05-23 16:33:22.491 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:33:22.492 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22eed10 - 1,2,3 [ CSSD]2013-05-23 16:33:22.492 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d75b0) proc(0x22eed10) pid(19626) version 11:1:1:4 [ CSSD]2013-05-23 16:33:22.499 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22eed10), client(1/0x22f2f20) [ CSSD]2013-05-23 16:33:22.499 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2f20) [ CSSD]2013-05-23 16:33:22.499 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2f20 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 16:33:22.499 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:22.499 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1610 (0x7f431bff1510) [ CSSD]2013-05-23 16:33:22.499 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1510 (RPC#1610) tag(64a002c) sent to node 1 [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(064a002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014648, gin 1, member 0 [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9bc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1510 (RPC#1610) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1510, ret 0, client 0x22f2f20 member 0x22eeb00 [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2f20 [ CSSD]2013-05-23 16:33:22.500 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1610 [ CSSD]2013-05-23 16:33:22.506 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2f20) [ CSSD]2013-05-23 16:33:22.506 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2f20), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:22.506 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eeb00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:22.506 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1611 (0x7f431bff1590) [ CSSD]2013-05-23 16:33:22.506 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1590 (RPC#1611) tag(64b002c) sent to node 1 [ CSSD]2013-05-23 16:33:22.507 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(064b002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:22.507 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:22.507 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1590 (RPC#1611) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:22.507 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1590, ret 0, client 0x22f2f20 member 0x22eeb00 [ CSSD]2013-05-23 16:33:22.507 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1611 [ CSSD]2013-05-23 16:33:22.507 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eeb00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:33:22.517 [1191196992] >TRACE: clssgmAllocProc: (0x2327c40) allocated [ CSSD]2013-05-23 16:33:22.518 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:33:22.519 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327c40 - 1,2,3 [ CSSD]2013-05-23 16:33:22.519 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e230) proc(0x2327c40) pid(19637) version 11:1:1:4 [ CSSD]2013-05-23 16:33:22.524 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2327c40), client(1/0x23283d0) [ CSSD]2013-05-23 16:33:22.524 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23283d0) [ CSSD]2013-05-23 16:33:22.524 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23283d0 with con 0x2327ec0, requested num -1 [ CSSD]2013-05-23 16:33:22.524 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:22.524 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1612 (0x7f431bff1610) [ CSSD]2013-05-23 16:33:22.525 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1610 (RPC#1612) tag(64c002c) sent to node 1 [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(064c002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014649, gin 1, member 0 [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1610 (RPC#1612) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1610, ret 0, client 0x23283d0 member 0x2327b20 [ CSSD]2013-05-23 16:33:22.525 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23283d0 [ CSSD]2013-05-23 16:33:22.526 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1612 [ CSSD]2013-05-23 16:33:22.532 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23283d0) [ CSSD]2013-05-23 16:33:22.532 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23283d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:22.532 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327b20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:22.532 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1613 (0x7f431bff1690) [ CSSD]2013-05-23 16:33:22.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1690 (RPC#1613) tag(64d002c) sent to node 1 [ CSSD]2013-05-23 16:33:22.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(064d002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:22.533 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:22.533 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1690 (RPC#1613) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:22.533 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1690, ret 0, client 0x23283d0 member 0x2327b20 [ CSSD]2013-05-23 16:33:22.533 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1613 [ CSSD]2013-05-23 16:33:22.533 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327b20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:33:22.710 [1191196992] >TRACE: clssgmDeadProc: proc 0x22eed10 [ CSSD]2013-05-23 16:33:22.710 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22eed10) con(0x22d75b0) skgpid ospid 19626 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:33:22.736 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327c40 [ CSSD]2013-05-23 16:33:22.736 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327c40) con(0x205e230) skgpid ospid 19637 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:33:22.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:22.840 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:22.901 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:33:22.902 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1614 (0x7f431bff1710) [ CSSD]2013-05-23 16:33:22.902 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1710 (RPC#1614) tag(64e002c) sent to node 1 [ CSSD]2013-05-23 16:33:22.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:22.902 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1614 (0x7f431bff1710) [ CSSD]2013-05-23 16:33:22.902 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1710 (RPC#1614) state 6, flags 0x202 [ CSSD]2013-05-23 16:33:22.902 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1710, ret 0, client (nil) [ CSSD]2013-05-23 16:33:22.903 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1614 [ CSSD]2013-05-23 16:33:24.307 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:24.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:24.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:26.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:26.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:26.899 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:26.963 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:26.963 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:28.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:28.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:28.905 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:30.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:30.914 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:30.914 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:31.966 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:31.966 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:32.398 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:32.751 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:32.751 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014650, gin 1, member 0 [ CSSD]2013-05-23 16:33:32.751 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:33:32.752 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:33:32.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:32.757 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:33:32.757 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205e300) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:33:32.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:32.920 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:32.926 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:33:32.926 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1615 (0x7f431bff1790) [ CSSD]2013-05-23 16:33:32.926 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1790 (RPC#1615) tag(64f002c) sent to node 1 [ CSSD]2013-05-23 16:33:32.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:32.926 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1615 (0x7f431bff1790) [ CSSD]2013-05-23 16:33:32.926 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1790 (RPC#1615) state 6, flags 0x202 [ CSSD]2013-05-23 16:33:32.926 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1790, ret 0, client (nil) [ CSSD]2013-05-23 16:33:32.927 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1615 [ CSSD]2013-05-23 16:33:34.429 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:34.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:34.943 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:36.435 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:36.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:36.949 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:36.970 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:36.970 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:37.767 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 16:33:37.768 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:33:37.769 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 16:33:37.769 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c11fd30) pid(19779) version 11:1:1:4 [ CSSD]2013-05-23 16:33:37.785 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:33:37.785 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:33:37.785 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:33:37.785 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:37.785 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1616 (0x7f431bff1810) [ CSSD]2013-05-23 16:33:37.785 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1810 (RPC#1616) tag(650002c) sent to node 1 [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0650002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014651, gin 1, member 0 [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1810 (RPC#1616) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1810, ret 0, client 0x7f431c109f80 member 0x7f431c12f520 [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:33:37.786 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1616 [ CSSD]2013-05-23 16:33:37.793 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:33:37.793 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:37.793 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:37.793 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1617 (0x7f431bff1890) [ CSSD]2013-05-23 16:33:37.793 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1890 (RPC#1617) tag(651002c) sent to node 1 [ CSSD]2013-05-23 16:33:37.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0651002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:37.794 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:37.794 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1890 (RPC#1617) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:37.794 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1890, ret 0, client 0x7f431c109f80 member 0x7f431c12f520 [ CSSD]2013-05-23 16:33:37.794 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1617 [ CSSD]2013-05-23 16:33:37.794 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:33:37.996 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 16:33:37.996 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c11ab50) skgpid ospid 19779 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:33:38.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:39.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:39.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:39.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:39.053 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014652, gin 1, member 0 [ CSSD]2013-05-23 16:33:39.053 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:33:39.053 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:33:39.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:39.060 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:33:39.060 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22d74f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:33:39.186 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:33:39.186 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:33:39.187 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:33:39.188 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:33:39.189 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:33:39.189 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:33:40.473 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:41.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:41.007 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:41.973 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:41.973 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:42.478 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:42.952 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:33:42.952 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1618 (0x7f431bff1910) [ CSSD]2013-05-23 16:33:42.952 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1910 (RPC#1618) tag(652002c) sent to node 1 [ CSSD]2013-05-23 16:33:42.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:42.953 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1618 (0x7f431bff1910) [ CSSD]2013-05-23 16:33:42.953 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1910 (RPC#1618) state 6, flags 0x202 [ CSSD]2013-05-23 16:33:42.953 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1910, ret 0, client (nil) [ CSSD]2013-05-23 16:33:42.953 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1618 [ CSSD]2013-05-23 16:33:43.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:43.012 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:44.489 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:45.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:45.024 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:46.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:46.976 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:46.976 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:47.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:47.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:48.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:48.004 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014653, gin 1, member 0 [ CSSD]2013-05-23 16:33:48.004 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:33:48.004 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:33:48.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:48.010 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:33:48.010 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d74f0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:33:48.507 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:49.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:49.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:50.531 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:51.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:51.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:51.979 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:51.979 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:52.534 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:52.978 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:33:52.979 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1619 (0x7f431bff1990) [ CSSD]2013-05-23 16:33:52.979 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1990 (RPC#1619) tag(653002c) sent to node 1 [ CSSD]2013-05-23 16:33:52.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:52.980 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1619 (0x7f431bff1990) [ CSSD]2013-05-23 16:33:52.980 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1990 (RPC#1619) state 6, flags 0x202 [ CSSD]2013-05-23 16:33:52.980 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1990, ret 0, client (nil) [ CSSD]2013-05-23 16:33:52.980 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1619 [ CSSD]2013-05-23 16:33:53.029 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:33:53.030 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:33:53.031 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:33:53.031 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c10a190) pid(19959) version 11:1:1:4 [ CSSD]2013-05-23 16:33:53.037 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c10a190), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:33:53.037 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:33:53.037 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:33:53.037 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:53.037 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1620 (0x7f431bff1a10) [ CSSD]2013-05-23 16:33:53.037 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1a10 (RPC#1620) tag(654002c) sent to node 1 [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0654002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014654, gin 1, member 0 [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1a10 (RPC#1620) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1a10, ret 0, client 0x7f431c109f80 member 0x7f431c12f520 [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:33:53.038 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1620 [ CSSD]2013-05-23 16:33:53.046 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:33:53.046 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:53.046 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:33:53.046 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1621 (0x7f431bff1a90) [ CSSD]2013-05-23 16:33:53.046 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1a90 (RPC#1621) tag(655002c) sent to node 1 [ CSSD]2013-05-23 16:33:53.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0655002c) incarnation(174631051) [ CSSD]2013-05-23 16:33:53.046 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:33:53.046 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1a90 (RPC#1621) state 6, flags 0x100 [ CSSD]2013-05-23 16:33:53.046 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1a90, ret 0, client 0x7f431c109f80 member 0x7f431c12f520 [ CSSD]2013-05-23 16:33:53.047 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1621 [ CSSD]2013-05-23 16:33:53.047 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:33:53.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:53.113 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:53.249 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:33:53.249 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e6720) skgpid ospid 19959 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:33:53.902 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22edf00) [ CSSD]2013-05-23 16:33:53.902 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22edf00), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:33:53.902 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22edf00), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:33:53.902 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:33:53.902 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed960 [ CSSD]2013-05-23 16:33:53.902 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed960) con(0x23184b0) skgpid 14469 ospid 14469 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:33:54.540 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:55.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:55.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:56.546 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:56.982 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:33:56.982 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:33:57.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:57.131 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:33:58.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:33:59.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:33:59.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:00.583 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:01.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:01.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:01.985 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:01.985 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:02.613 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:03.004 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:34:03.005 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1622 (0x7f431bff1b10) [ CSSD]2013-05-23 16:34:03.005 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1b10 (RPC#1622) tag(656002c) sent to node 1 [ CSSD]2013-05-23 16:34:03.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:03.005 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1622 (0x7f431bff1b10) [ CSSD]2013-05-23 16:34:03.005 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1b10 (RPC#1622) state 6, flags 0x202 [ CSSD]2013-05-23 16:34:03.005 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1b10, ret 0, client (nil) [ CSSD]2013-05-23 16:34:03.005 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1622 [ CSSD]2013-05-23 16:34:03.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:03.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:03.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:03.273 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014655, gin 1, member 0 [ CSSD]2013-05-23 16:34:03.273 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:34:03.273 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:34:03.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:03.280 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:34:03.280 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22edf50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:34:04.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:05.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:05.245 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:05.988 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:05.988 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:34:06.624 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:07.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:07.250 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:08.290 [1191196992] >TRACE: clssgmAllocProc: (0x22ee920) allocated [ CSSD]2013-05-23 16:34:08.291 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:34:08.292 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee920 - 1,2,3 [ CSSD]2013-05-23 16:34:08.292 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4f40) proc(0x22ee920) pid(20112) version 11:1:1:4 [ CSSD]2013-05-23 16:34:08.297 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee920), client(1/0x22f2a10) [ CSSD]2013-05-23 16:34:08.297 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:08.297 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2a10 with con 0x22f3ee0, requested num -1 [ CSSD]2013-05-23 16:34:08.297 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:08.297 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1623 (0x7f431bff1b90) [ CSSD]2013-05-23 16:34:08.297 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1b90 (RPC#1623) tag(657002c) sent to node 1 [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0657002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014656, gin 1, member 0 [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f2b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1b90 (RPC#1623) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1b90, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2a10 [ CSSD]2013-05-23 16:34:08.298 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1623 [ CSSD]2013-05-23 16:34:08.304 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:08.304 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2a10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:08.304 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:08.304 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1624 (0x7f431bff1c10) [ CSSD]2013-05-23 16:34:08.304 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1c10 (RPC#1624) tag(658002c) sent to node 1 [ CSSD]2013-05-23 16:34:08.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0658002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:08.304 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:08.304 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1c10 (RPC#1624) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:08.304 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1c10, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:08.305 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1624 [ CSSD]2013-05-23 16:34:08.305 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:34:08.507 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee920 [ CSSD]2013-05-23 16:34:08.507 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee920) con(0x22f4f40) skgpid ospid 20112 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:34:08.678 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:09.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:09.302 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:10.684 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:10.991 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:10.991 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:11.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:11.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:12.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:13.030 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:34:13.030 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1625 (0x7f431bff1c90) [ CSSD]2013-05-23 16:34:13.030 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1c90 (RPC#1625) tag(659002c) sent to node 1 [ CSSD]2013-05-23 16:34:13.031 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:13.031 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1625 (0x7f431bff1c90) [ CSSD]2013-05-23 16:34:13.031 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1c90 (RPC#1625) state 6, flags 0x202 [ CSSD]2013-05-23 16:34:13.031 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1c90, ret 0, client (nil) [ CSSD]2013-05-23 16:34:13.031 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1625 [ CSSD]2013-05-23 16:34:13.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:13.313 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:14.721 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:15.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:15.325 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:15.994 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:15.995 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:16.727 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:17.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:17.330 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:18.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:18.523 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014657, gin 1, member 0 [ CSSD]2013-05-23 16:34:18.523 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:34:18.523 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:34:18.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:18.529 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:34:18.529 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22edf50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:34:18.778 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:18.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:18.953 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014658, gin 1, member 0 [ CSSD]2013-05-23 16:34:18.953 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:34:18.953 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:34:18.960 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:18.960 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:34:18.960 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22edf50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:34:19.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:19.343 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:20.783 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:20.997 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:20.998 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:21.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:21.349 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:22.749 [1191196992] >TRACE: clssgmAllocProc: (0x22ee920) allocated [ CSSD]2013-05-23 16:34:22.750 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:34:22.750 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee920 - 1,2,3 [ CSSD]2013-05-23 16:34:22.750 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4f40) proc(0x22ee920) pid(20248) version 11:1:1:4 [ CSSD]2013-05-23 16:34:22.756 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee920), client(1/0x22f2a10) [ CSSD]2013-05-23 16:34:22.756 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:22.756 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2a10 with con 0x22f3ee0, requested num -1 [ CSSD]2013-05-23 16:34:22.756 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:22.756 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1626 (0x7f431bff1d10) [ CSSD]2013-05-23 16:34:22.757 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1d10 (RPC#1626) tag(65a002c) sent to node 1 [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(065a002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014659, gin 1, member 0 [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f2b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1d10 (RPC#1626) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1d10, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:22.757 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2a10 [ CSSD]2013-05-23 16:34:22.758 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1626 [ CSSD]2013-05-23 16:34:22.764 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:22.764 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2a10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:22.764 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:22.764 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1627 (0x7f431bff1d90) [ CSSD]2013-05-23 16:34:22.764 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1d90 (RPC#1627) tag(65b002c) sent to node 1 [ CSSD]2013-05-23 16:34:22.764 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(065b002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:22.764 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:22.764 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1d90 (RPC#1627) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:22.764 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1d90, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:22.765 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1627 [ CSSD]2013-05-23 16:34:22.765 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:34:22.796 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:22.967 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee920 [ CSSD]2013-05-23 16:34:22.967 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee920) con(0x22f4f40) skgpid ospid 20248 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:34:23.050 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:34:23.050 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1628 (0x7f431bff1e10) [ CSSD]2013-05-23 16:34:23.051 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1e10 (RPC#1628) tag(65c002c) sent to node 1 [ CSSD]2013-05-23 16:34:23.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:23.051 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1628 (0x7f431bff1e10) [ CSSD]2013-05-23 16:34:23.051 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1e10 (RPC#1628) state 6, flags 0x202 [ CSSD]2013-05-23 16:34:23.051 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1e10, ret 0, client (nil) [ CSSD]2013-05-23 16:34:23.051 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1628 [ CSSD]2013-05-23 16:34:23.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:23.354 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:23.538 [1191196992] >TRACE: clssgmAllocProc: (0x22ee920) allocated [ CSSD]2013-05-23 16:34:23.540 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:34:23.540 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee920 - 1,2,3 [ CSSD]2013-05-23 16:34:23.540 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4f40) proc(0x22ee920) pid(20265) version 11:1:1:4 [ CSSD]2013-05-23 16:34:23.556 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee920), client(1/0x22f2a10) [ CSSD]2013-05-23 16:34:23.556 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:23.556 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2a10 with con 0x22f3ee0, requested num -1 [ CSSD]2013-05-23 16:34:23.556 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:23.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1629 (0x7f431bff1e90) [ CSSD]2013-05-23 16:34:23.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1e90 (RPC#1629) tag(65d002c) sent to node 1 [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(065d002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014660, gin 1, member 0 [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f2b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1e90 (RPC#1629) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff1e90, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2a10 [ CSSD]2013-05-23 16:34:23.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1629 [ CSSD]2013-05-23 16:34:23.564 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:23.564 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2a10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:23.564 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:23.564 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1630 (0x7f431bff1f10) [ CSSD]2013-05-23 16:34:23.564 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1f10 (RPC#1630) tag(65e002c) sent to node 1 [ CSSD]2013-05-23 16:34:23.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(065e002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:23.564 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:23.564 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1f10 (RPC#1630) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:23.564 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff1f10, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:23.565 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1630 [ CSSD]2013-05-23 16:34:23.565 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:34:23.767 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee920 [ CSSD]2013-05-23 16:34:23.767 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee920) con(0x22f4f40) skgpid ospid 20265 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:34:24.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:25.000 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:25.000 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:34:25.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:25.406 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:26.011 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:34:26.012 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:34:26.013 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:34:26.013 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:34:26.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:26.098 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014661, gin 1, member 0 [ CSSD]2013-05-23 16:34:26.098 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f37a0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:34:26.098 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:34:26.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:26.105 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:34:26.105 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22edf50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:34:26.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:27.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:27.432 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:28.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:29.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:29.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:30.003 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:30.003 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:30.902 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:31.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:31.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:32.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:33.080 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:34:33.081 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1631 (0x7f431bff1f90) [ CSSD]2013-05-23 16:34:33.081 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff1f90 (RPC#1631) tag(65f002c) sent to node 1 [ CSSD]2013-05-23 16:34:33.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:33.081 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1631 (0x7f431bff1f90) [ CSSD]2013-05-23 16:34:33.081 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff1f90 (RPC#1631) state 6, flags 0x202 [ CSSD]2013-05-23 16:34:33.081 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff1f90, ret 0, client (nil) [ CSSD]2013-05-23 16:34:33.082 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1631 [ CSSD]2013-05-23 16:34:33.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:33.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:33.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:33.776 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014662, gin 1, member 0 [ CSSD]2013-05-23 16:34:33.776 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:34:33.776 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:34:33.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:33.783 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:34:33.783 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22edf50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:34:34.921 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:35.005 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:35.005 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:35.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:35.527 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:36.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:37.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:37.533 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:38.809 [1191196992] >TRACE: clssgmAllocProc: (0x22ee920) allocated [ CSSD]2013-05-23 16:34:38.810 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:34:38.810 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee920 - 1,2,3 [ CSSD]2013-05-23 16:34:38.810 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4f40) proc(0x22ee920) pid(20400) version 11:1:1:4 [ CSSD]2013-05-23 16:34:38.816 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee920), client(1/0x22f2a10) [ CSSD]2013-05-23 16:34:38.816 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:38.816 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2a10 with con 0x22f3ee0, requested num -1 [ CSSD]2013-05-23 16:34:38.816 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:38.816 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1632 (0x7f431bff2010) [ CSSD]2013-05-23 16:34:38.816 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2010 (RPC#1632) tag(660002c) sent to node 1 [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0660002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014663, gin 1, member 0 [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f2b80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2010 (RPC#1632) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2010, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2a10 [ CSSD]2013-05-23 16:34:38.817 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1632 [ CSSD]2013-05-23 16:34:38.823 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:34:38.823 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2a10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:38.823 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:38.823 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1633 (0x7f431bff2090) [ CSSD]2013-05-23 16:34:38.823 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2090 (RPC#1633) tag(661002c) sent to node 1 [ CSSD]2013-05-23 16:34:38.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0661002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:38.823 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:38.823 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2090 (RPC#1633) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:38.823 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2090, ret 0, client 0x22f2a10 member 0x22f2e80 [ CSSD]2013-05-23 16:34:38.824 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1633 [ CSSD]2013-05-23 16:34:38.824 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:34:38.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:39.026 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee920 [ CSSD]2013-05-23 16:34:39.026 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee920) con(0x22f4f40) skgpid ospid 20400 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:34:39.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:39.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:40.008 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:40.008 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:40.985 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:41.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:41.630 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:42.997 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:43.110 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:34:43.111 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1634 (0x7f431bff2110) [ CSSD]2013-05-23 16:34:43.111 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2110 (RPC#1634) tag(662002c) sent to node 1 [ CSSD]2013-05-23 16:34:43.111 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:43.111 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1634 (0x7f431bff2110) [ CSSD]2013-05-23 16:34:43.111 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2110 (RPC#1634) state 6, flags 0x202 [ CSSD]2013-05-23 16:34:43.111 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2110, ret 0, client (nil) [ CSSD]2013-05-23 16:34:43.111 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1634 [ CSSD]2013-05-23 16:34:43.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:43.636 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:45.002 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:45.011 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:45.011 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:45.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:45.647 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:47.026 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:47.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:47.653 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:49.013 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:49.013 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:34:49.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:49.028 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014664, gin 1, member 0 [ CSSD]2013-05-23 16:34:49.028 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:34:49.028 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:34:49.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:49.036 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:34:49.036 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22edf50) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:34:49.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:49.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:49.693 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:50.910 [1191196992] >TRACE: clssgmAllocProc: (0x22ee920) allocated [ CSSD]2013-05-23 16:34:50.910 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:34:50.911 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee920 - 1,2,3 [ CSSD]2013-05-23 16:34:50.911 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4f40) proc(0x22ee920) pid(20007/20007) version 11:1:1:4 [ CSSD]2013-05-23 16:34:50.916 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee920), client(1/0x22f3ef0) [ CSSD]2013-05-23 16:34:50.916 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 16:34:50.916 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:34:50.916 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:34:51.104 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:51.124 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee920), client(2/0x22f2e60) [ CSSD]2013-05-23 16:34:51.124 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f2e60) [ CSSD]2013-05-23 16:34:51.124 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:34:51.124 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 16:34:51.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:51.699 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:53.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:53.141 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:34:53.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1635 (0x7f431bff2190) [ CSSD]2013-05-23 16:34:53.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2190 (RPC#1635) tag(663002c) sent to node 1 [ CSSD]2013-05-23 16:34:53.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:53.142 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1635 (0x7f431bff2190) [ CSSD]2013-05-23 16:34:53.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2190 (RPC#1635) state 6, flags 0x202 [ CSSD]2013-05-23 16:34:53.142 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2190, ret 0, client (nil) [ CSSD]2013-05-23 16:34:53.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1635 [ CSSD]2013-05-23 16:34:53.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:53.704 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:54.016 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:54.016 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:54.059 [1191196992] >TRACE: clssgmAllocProc: (0x22ecec0) allocated [ CSSD]2013-05-23 16:34:54.060 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:34:54.061 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ecec0 - 1,2,3 [ CSSD]2013-05-23 16:34:54.061 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2d20) proc(0x22ecec0) pid(20549) version 11:1:1:4 [ CSSD]2013-05-23 16:34:54.066 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ecec0), client(1/0x22ed5a0) [ CSSD]2013-05-23 16:34:54.066 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ed5a0) [ CSSD]2013-05-23 16:34:54.066 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ed5a0 with con 0x2319230, requested num -1 [ CSSD]2013-05-23 16:34:54.066 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:54.066 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1636 (0x7f431bff2210) [ CSSD]2013-05-23 16:34:54.067 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2210 (RPC#1636) tag(664002c) sent to node 1 [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0664002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014665, gin 1, member 0 [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed710/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2210 (RPC#1636) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2210, ret 0, client 0x22ed5a0 member 0x22eda10 [ CSSD]2013-05-23 16:34:54.067 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ed5a0 [ CSSD]2013-05-23 16:34:54.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1636 [ CSSD]2013-05-23 16:34:54.074 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ed5a0) [ CSSD]2013-05-23 16:34:54.074 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ed5a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:54.074 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eda10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:34:54.074 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1637 (0x7f431bff2290) [ CSSD]2013-05-23 16:34:54.074 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2290 (RPC#1637) tag(665002c) sent to node 1 [ CSSD]2013-05-23 16:34:54.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0665002c) incarnation(174631051) [ CSSD]2013-05-23 16:34:54.074 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:34:54.074 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2290 (RPC#1637) state 6, flags 0x100 [ CSSD]2013-05-23 16:34:54.074 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2290, ret 0, client 0x22ed5a0 member 0x22eda10 [ CSSD]2013-05-23 16:34:54.074 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1637 [ CSSD]2013-05-23 16:34:54.074 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eda10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:34:54.277 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ecec0 [ CSSD]2013-05-23 16:34:54.277 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ecec0) con(0x22f2d20) skgpid ospid 20549 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:34:55.123 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:55.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:55.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:57.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:57.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:57.723 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:34:59.019 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:34:59.020 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:34:59.180 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:34:59.364 [1191196992] >TRACE: clssgmAllocProc: (0x23191c0) allocated [ CSSD]2013-05-23 16:34:59.365 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:34:59.365 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23191c0 - 1,2,3 [ CSSD]2013-05-23 16:34:59.365 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9fa0) proc(0x23191c0) pid(19309/19309) version 11:1:1:4 [ CSSD]2013-05-23 16:34:59.382 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x23191c0), client(1/0x22ed590) [ CSSD]2013-05-23 16:34:59.382 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ed590) [ CSSD]2013-05-23 16:34:59.382 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:34:59.382 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 16:34:59.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:34:59.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:01.185 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:01.379 [1191196992] >TRACE: clssgmAllocProc: (0x23286e0) allocated [ CSSD]2013-05-23 16:35:01.380 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:35:01.380 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23286e0 - 1,2,3 [ CSSD]2013-05-23 16:35:01.380 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2326460) proc(0x23286e0) pid(19311/19311) version 11:1:1:4 [ CSSD]2013-05-23 16:35:01.396 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23286e0), client(1/0x2328d60) [ CSSD]2013-05-23 16:35:01.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2328d60) [ CSSD]2013-05-23 16:35:01.396 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:35:01.396 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 16:35:01.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:01.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:03.171 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:35:03.171 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1638 (0x7f431bff2310) [ CSSD]2013-05-23 16:35:03.172 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2310 (RPC#1638) tag(666002c) sent to node 1 [ CSSD]2013-05-23 16:35:03.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:03.172 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1638 (0x7f431bff2310) [ CSSD]2013-05-23 16:35:03.172 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2310 (RPC#1638) state 6, flags 0x202 [ CSSD]2013-05-23 16:35:03.172 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2310, ret 0, client (nil) [ CSSD]2013-05-23 16:35:03.172 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1638 [ CSSD]2013-05-23 16:35:03.196 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:03.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:03.745 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:04.023 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:04.023 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:04.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:04.278 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014666, gin 1, member 0 [ CSSD]2013-05-23 16:35:04.278 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:35:04.278 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:35:04.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:04.285 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:35:04.285 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23292c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:35:05.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:05.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:05.738 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:07.247 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:07.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:07.744 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:08.429 [1191196992] >TRACE: clssgmAllocProc: (0x23117a0) allocated [ CSSD]2013-05-23 16:35:08.430 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:35:08.430 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23117a0 - 1,2,3 [ CSSD]2013-05-23 16:35:08.430 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x23117a0) pid(20714) version 11:1:1:4 [ CSSD]2013-05-23 16:35:09.001 [1191196992] >TRACE: clssgmDeadProc: proc 0x23117a0 [ CSSD]2013-05-23 16:35:09.001 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23117a0) con(0x22f2a10) skgpid ospid 20714 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:09.026 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:09.026 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:09.281 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:09.307 [1191196992] >TRACE: clssgmAllocProc: (0x2311850) allocated [ CSSD]2013-05-23 16:35:09.308 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:35:09.309 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2311850 - 1,2,3 [ CSSD]2013-05-23 16:35:09.309 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f5c0) proc(0x2311850) pid(20823) version 11:1:1:4 [ CSSD]2013-05-23 16:35:09.314 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2311850), client(1/0x23117b0) [ CSSD]2013-05-23 16:35:09.314 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23117b0) [ CSSD]2013-05-23 16:35:09.314 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23117b0 with con 0x2311b40, requested num -1 [ CSSD]2013-05-23 16:35:09.314 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:09.314 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1639 (0x7f431bff2390) [ CSSD]2013-05-23 16:35:09.314 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2390 (RPC#1639) tag(667002c) sent to node 1 [ CSSD]2013-05-23 16:35:09.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0667002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:09.314 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014667, gin 1, member 0 [ CSSD]2013-05-23 16:35:09.314 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2312130/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:35:09.315 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:35:09.315 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:35:09.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2390 (RPC#1639) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:09.315 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2390, ret 0, client 0x23117b0 member 0x2312430 [ CSSD]2013-05-23 16:35:09.315 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23117b0 [ CSSD]2013-05-23 16:35:09.315 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1639 [ CSSD]2013-05-23 16:35:09.320 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23117b0) [ CSSD]2013-05-23 16:35:09.320 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23117b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:09.320 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2312430) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:09.320 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1640 (0x7f431bff2410) [ CSSD]2013-05-23 16:35:09.320 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2410 (RPC#1640) tag(668002c) sent to node 1 [ CSSD]2013-05-23 16:35:09.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0668002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:09.320 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:09.320 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2410 (RPC#1640) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:09.320 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2410, ret 0, client 0x23117b0 member 0x2312430 [ CSSD]2013-05-23 16:35:09.321 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1640 [ CSSD]2013-05-23 16:35:09.321 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2312430) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:35:09.523 [1191196992] >TRACE: clssgmDeadProc: proc 0x2311850 [ CSSD]2013-05-23 16:35:09.523 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2311850) con(0x230f5c0) skgpid ospid 20823 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:09.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:09.756 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:11.286 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:11.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:11.762 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:13.190 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:35:13.191 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1641 (0x7f431bff2490) [ CSSD]2013-05-23 16:35:13.191 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2490 (RPC#1641) tag(669002c) sent to node 1 [ CSSD]2013-05-23 16:35:13.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:13.191 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1641 (0x7f431bff2490) [ CSSD]2013-05-23 16:35:13.191 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2490 (RPC#1641) state 6, flags 0x202 [ CSSD]2013-05-23 16:35:13.191 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2490, ret 0, client (nil) [ CSSD]2013-05-23 16:35:13.192 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1641 [ CSSD]2013-05-23 16:35:13.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:13.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:13.768 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:14.029 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:14.029 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:15.334 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:15.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:15.819 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:17.340 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:17.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:17.825 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:19.032 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:19.032 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:19.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:19.209 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014668, gin 1, member 0 [ CSSD]2013-05-23 16:35:19.209 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:35:19.209 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:35:19.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:19.215 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:35:19.215 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23292c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:35:19.336 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e9100) [ CSSD]2013-05-23 16:35:19.336 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e9100), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:35:19.336 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e9100), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:35:19.336 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 16:35:19.336 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8bd0 [ CSSD]2013-05-23 16:35:19.336 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8bd0) con(0x23224d0) skgpid 15653 ospid 15653 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:19.391 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:19.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:19.531 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014669, gin 1, member 0 [ CSSD]2013-05-23 16:35:19.531 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:35:19.531 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:35:19.538 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:19.538 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:35:19.538 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23291d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:35:19.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:19.835 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:21.397 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:21.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:21.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:23.014 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:35:23.014 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:35:23.015 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:35:23.015 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8310) pid(20962) version 11:1:1:4 [ CSSD]2013-05-23 16:35:23.021 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x2302db0) [ CSSD]2013-05-23 16:35:23.021 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302db0) [ CSSD]2013-05-23 16:35:23.021 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302db0 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:35:23.021 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:23.021 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1642 (0x7f431bff2510) [ CSSD]2013-05-23 16:35:23.021 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2510 (RPC#1642) tag(66a002c) sent to node 1 [ CSSD]2013-05-23 16:35:23.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(066a002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:23.021 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014670, gin 1, member 0 [ CSSD]2013-05-23 16:35:23.021 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:35:23.022 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:35:23.022 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:35:23.022 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2510 (RPC#1642) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:23.022 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2510, ret 0, client 0x2302db0 member 0x22e81f0 [ CSSD]2013-05-23 16:35:23.022 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302db0 [ CSSD]2013-05-23 16:35:23.022 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1642 [ CSSD]2013-05-23 16:35:23.029 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302db0) [ CSSD]2013-05-23 16:35:23.029 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302db0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:23.029 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:23.029 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1643 (0x7f431bff2590) [ CSSD]2013-05-23 16:35:23.029 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2590 (RPC#1643) tag(66b002c) sent to node 1 [ CSSD]2013-05-23 16:35:23.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(066b002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:23.029 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:23.029 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2590 (RPC#1643) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:23.029 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2590, ret 0, client 0x2302db0 member 0x22e81f0 [ CSSD]2013-05-23 16:35:23.030 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1643 [ CSSD]2013-05-23 16:35:23.030 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e81f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:35:23.231 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:35:23.231 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1644 (0x7f431bff2610) [ CSSD]2013-05-23 16:35:23.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2610 (RPC#1644) tag(66c002c) sent to node 1 [ CSSD]2013-05-23 16:35:23.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:23.232 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1644 (0x7f431bff2610) [ CSSD]2013-05-23 16:35:23.232 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2610 (RPC#1644) state 6, flags 0x202 [ CSSD]2013-05-23 16:35:23.232 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2610, ret 0, client (nil) [ CSSD]2013-05-23 16:35:23.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1644 [ CSSD]2013-05-23 16:35:23.232 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:35:23.232 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x22f2a10) skgpid ospid 20962 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:23.407 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:23.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:23.865 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:24.035 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:24.035 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:24.553 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:35:24.554 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:35:24.554 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:35:24.554 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8310) pid(20985) version 11:1:1:4 [ CSSD]2013-05-23 16:35:24.560 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x2302db0) [ CSSD]2013-05-23 16:35:24.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302db0) [ CSSD]2013-05-23 16:35:24.560 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302db0 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:35:24.560 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:24.560 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1645 (0x7f431bff2690) [ CSSD]2013-05-23 16:35:24.560 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2690 (RPC#1645) tag(66d002c) sent to node 1 [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(066d002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014671, gin 1, member 0 [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2690 (RPC#1645) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2690, ret 0, client 0x2302db0 member 0x22e81f0 [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302db0 [ CSSD]2013-05-23 16:35:24.561 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1645 [ CSSD]2013-05-23 16:35:24.567 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302db0) [ CSSD]2013-05-23 16:35:24.567 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302db0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:24.567 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:24.567 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1646 (0x7f431bff2710) [ CSSD]2013-05-23 16:35:24.567 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2710 (RPC#1646) tag(66e002c) sent to node 1 [ CSSD]2013-05-23 16:35:24.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(066e002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:24.567 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:24.567 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2710 (RPC#1646) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:24.568 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2710, ret 0, client 0x2302db0 member 0x22e81f0 [ CSSD]2013-05-23 16:35:24.568 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1646 [ CSSD]2013-05-23 16:35:24.568 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e81f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:35:24.770 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:35:24.770 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x22f2a10) skgpid ospid 20985 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:25.413 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:25.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:25.919 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:27.418 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:27.924 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:27.924 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:29.038 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:29.039 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:29.432 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:29.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:29.937 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:31.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:31.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:31.962 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:33.041 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:33.041 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:35:33.251 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:35:33.251 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1647 (0x7f431bff2790) [ CSSD]2013-05-23 16:35:33.251 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2790 (RPC#1647) tag(66f002c) sent to node 1 [ CSSD]2013-05-23 16:35:33.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:33.252 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1647 (0x7f431bff2790) [ CSSD]2013-05-23 16:35:33.252 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2790 (RPC#1647) state 6, flags 0x202 [ CSSD]2013-05-23 16:35:33.252 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2790, ret 0, client (nil) [ CSSD]2013-05-23 16:35:33.252 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1647 [ CSSD]2013-05-23 16:35:33.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:34.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:34.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:34.782 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:34.782 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014672, gin 1, member 0 [ CSSD]2013-05-23 16:35:34.782 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:35:34.782 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:35:34.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:34.789 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:35:34.789 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23291d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:35:35.515 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:36.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:36.020 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:37.521 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:38.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:38.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:38.044 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:38.044 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:39.532 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:39.801 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:35:39.802 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:35:39.802 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:35:39.802 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302db0) proc(0x22e8310) pid(21152) version 11:1:1:4 [ CSSD]2013-05-23 16:35:39.808 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x22e8a60) [ CSSD]2013-05-23 16:35:39.808 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8a60) [ CSSD]2013-05-23 16:35:39.808 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8a60 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:35:39.808 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:39.808 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1648 (0x7f431bff2810) [ CSSD]2013-05-23 16:35:39.809 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2810 (RPC#1648) tag(670002c) sent to node 1 [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0670002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014673, gin 1, member 0 [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8bd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2810 (RPC#1648) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2810, ret 0, client 0x22e8a60 member 0x22e81d0 [ CSSD]2013-05-23 16:35:39.809 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8a60 [ CSSD]2013-05-23 16:35:39.810 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1648 [ CSSD]2013-05-23 16:35:39.815 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8a60) [ CSSD]2013-05-23 16:35:39.815 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8a60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:39.815 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:39.815 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1649 (0x7f431bff2890) [ CSSD]2013-05-23 16:35:39.815 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2890 (RPC#1649) tag(671002c) sent to node 1 [ CSSD]2013-05-23 16:35:39.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0671002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:39.816 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:39.816 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2890 (RPC#1649) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:39.816 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2890, ret 0, client 0x22e8a60 member 0x22e81d0 [ CSSD]2013-05-23 16:35:39.816 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1649 [ CSSD]2013-05-23 16:35:39.816 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e81d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:35:40.018 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:35:40.018 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x2302db0) skgpid ospid 21152 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:40.037 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:40.037 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:41.538 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:42.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:42.063 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:43.047 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:43.047 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:43.272 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:35:43.272 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1650 (0x7f431bff2910) [ CSSD]2013-05-23 16:35:43.272 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2910 (RPC#1650) tag(672002c) sent to node 1 [ CSSD]2013-05-23 16:35:43.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:43.273 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1650 (0x7f431bff2910) [ CSSD]2013-05-23 16:35:43.273 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2910 (RPC#1650) state 6, flags 0x202 [ CSSD]2013-05-23 16:35:43.273 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2910, ret 0, client (nil) [ CSSD]2013-05-23 16:35:43.273 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1650 [ CSSD]2013-05-23 16:35:43.590 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:44.114 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:44.114 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:45.595 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:46.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:46.120 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:47.607 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:48.050 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:48.050 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:48.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:48.145 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:49.613 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:50.042 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:50.042 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014674, gin 1, member 0 [ CSSD]2013-05-23 16:35:50.042 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:35:50.042 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:35:50.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:50.049 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:35:50.049 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23291d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:35:50.238 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:50.238 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:51.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:52.052 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:52.053 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:35:52.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:52.244 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:53.302 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:35:53.302 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1651 (0x7f431bff2990) [ CSSD]2013-05-23 16:35:53.303 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2990 (RPC#1651) tag(673002c) sent to node 1 [ CSSD]2013-05-23 16:35:53.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:53.303 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1651 (0x7f431bff2990) [ CSSD]2013-05-23 16:35:53.303 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2990 (RPC#1651) state 6, flags 0x202 [ CSSD]2013-05-23 16:35:53.303 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2990, ret 0, client (nil) [ CSSD]2013-05-23 16:35:53.303 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1651 [ CSSD]2013-05-23 16:35:53.689 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:54.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:54.255 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:55.060 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:35:55.061 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:35:55.061 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:35:55.061 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302db0) proc(0x22e8310) pid(21298) version 11:1:1:4 [ CSSD]2013-05-23 16:35:55.067 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x22f2a10) [ CSSD]2013-05-23 16:35:55.067 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:35:55.067 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2a10 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:35:55.067 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:55.067 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1652 (0x7f431bff2a10) [ CSSD]2013-05-23 16:35:55.068 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2a10 (RPC#1652) tag(674002c) sent to node 1 [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0674002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014675, gin 1, member 0 [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2a10 (RPC#1652) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2a10, ret 0, client 0x22f2a10 member 0x22e81f0 [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2a10 [ CSSD]2013-05-23 16:35:55.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1652 [ CSSD]2013-05-23 16:35:55.074 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2a10) [ CSSD]2013-05-23 16:35:55.074 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2a10), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:55.074 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:35:55.075 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1653 (0x7f431bff2a90) [ CSSD]2013-05-23 16:35:55.075 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2a90 (RPC#1653) tag(675002c) sent to node 1 [ CSSD]2013-05-23 16:35:55.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0675002c) incarnation(174631051) [ CSSD]2013-05-23 16:35:55.075 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:35:55.075 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2a90 (RPC#1653) state 6, flags 0x100 [ CSSD]2013-05-23 16:35:55.075 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2a90, ret 0, client 0x22f2a10 member 0x22e81f0 [ CSSD]2013-05-23 16:35:55.075 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1653 [ CSSD]2013-05-23 16:35:55.075 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e81f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:35:55.278 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:35:55.278 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x2302db0) skgpid ospid 21298 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:35:55.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:56.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:56.261 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:57.055 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:35:57.055 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:35:57.707 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:35:58.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:35:58.287 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:35:59.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:00.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:00.338 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:01.739 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:02.058 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:02.058 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:02.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:02.344 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:03.332 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:36:03.332 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1654 (0x7f431bff2b10) [ CSSD]2013-05-23 16:36:03.333 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2b10 (RPC#1654) tag(676002c) sent to node 1 [ CSSD]2013-05-23 16:36:03.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:03.333 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1654 (0x7f431bff2b10) [ CSSD]2013-05-23 16:36:03.333 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2b10 (RPC#1654) state 6, flags 0x202 [ CSSD]2013-05-23 16:36:03.333 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2b10, ret 0, client (nil) [ CSSD]2013-05-23 16:36:03.333 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1654 [ CSSD]2013-05-23 16:36:03.811 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:04.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:04.355 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:05.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:05.313 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014676, gin 1, member 0 [ CSSD]2013-05-23 16:36:05.313 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:36:05.313 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:36:05.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:05.320 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:36:05.320 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23291d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:36:05.837 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:06.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:06.360 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:07.061 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:07.061 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:07.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:08.387 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:08.387 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:09.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:10.310 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:36:10.310 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:36:10.311 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:36:10.311 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8310) pid(21442) version 11:1:1:4 [ CSSD]2013-05-23 16:36:10.328 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x22e8a60) [ CSSD]2013-05-23 16:36:10.328 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8a60) [ CSSD]2013-05-23 16:36:10.328 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8a60 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:36:10.328 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:10.328 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1655 (0x7f431bff2b90) [ CSSD]2013-05-23 16:36:10.328 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2b90 (RPC#1655) tag(677002c) sent to node 1 [ CSSD]2013-05-23 16:36:10.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0677002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:10.328 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014677, gin 1, member 0 [ CSSD]2013-05-23 16:36:10.328 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8bd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:36:10.329 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:36:10.329 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:36:10.329 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2b90 (RPC#1655) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:10.329 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2b90, ret 0, client 0x22e8a60 member 0x22e81d0 [ CSSD]2013-05-23 16:36:10.329 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8a60 [ CSSD]2013-05-23 16:36:10.329 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1655 [ CSSD]2013-05-23 16:36:10.336 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8a60) [ CSSD]2013-05-23 16:36:10.336 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8a60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:10.336 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:10.336 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1656 (0x7f431bff2c10) [ CSSD]2013-05-23 16:36:10.336 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2c10 (RPC#1656) tag(678002c) sent to node 1 [ CSSD]2013-05-23 16:36:10.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0678002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:10.336 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:10.336 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2c10 (RPC#1656) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:10.336 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2c10, ret 0, client 0x22e8a60 member 0x22e81d0 [ CSSD]2013-05-23 16:36:10.337 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1656 [ CSSD]2013-05-23 16:36:10.337 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e81d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:36:10.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:10.429 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:10.539 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:36:10.539 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x22f2a10) skgpid ospid 21442 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:11.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:12.064 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:12.064 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:12.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:12.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:13.352 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:36:13.352 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1657 (0x7f431bff2c90) [ CSSD]2013-05-23 16:36:13.352 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2c90 (RPC#1657) tag(679002c) sent to node 1 [ CSSD]2013-05-23 16:36:13.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:13.353 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1657 (0x7f431bff2c90) [ CSSD]2013-05-23 16:36:13.353 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2c90 (RPC#1657) state 6, flags 0x202 [ CSSD]2013-05-23 16:36:13.353 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2c90, ret 0, client (nil) [ CSSD]2013-05-23 16:36:13.353 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1657 [ CSSD]2013-05-23 16:36:13.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:14.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:14.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:15.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:16.066 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:16.066 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:36:16.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:16.551 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:17.988 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:18.210 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:36:18.211 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:36:18.211 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:36:18.211 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302db0) proc(0x22e8310) pid(21542) version 11:1:1:4 [ CSSD]2013-05-23 16:36:18.228 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x22e8a60) [ CSSD]2013-05-23 16:36:18.228 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8a60) [ CSSD]2013-05-23 16:36:18.228 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22e8a60 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:36:18.228 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:36:18.228 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1658 (0x7f431bff2d10) [ CSSD]2013-05-23 16:36:18.228 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2d10 (RPC#1658) tag(67a002c) sent to node 1 [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(067a002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014678, gin 1, member 0 [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8bd0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2d10 (RPC#1658) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2d10, ret 0, client 0x22e8a60 member 0x22e81d0 [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8a60 [ CSSD]2013-05-23 16:36:18.229 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1658 [ CSSD]2013-05-23 16:36:18.236 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8a60) [ CSSD]2013-05-23 16:36:18.236 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8a60), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:36:18.236 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81d0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:36:18.236 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1659 (0x7f431bff2d90) [ CSSD]2013-05-23 16:36:18.236 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2d90 (RPC#1659) tag(67b002c) sent to node 1 [ CSSD]2013-05-23 16:36:18.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(067b002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:18.236 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:36:18.236 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2d90 (RPC#1659) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:18.236 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2d90, ret 0, client 0x22e8a60 member 0x22e81d0 [ CSSD]2013-05-23 16:36:18.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1659 [ CSSD]2013-05-23 16:36:18.237 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22e81d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:36:18.348 [1191196992] >TRACE: clssgmAllocProc: (0x2311420) allocated [ CSSD]2013-05-23 16:36:18.349 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:36:18.349 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2311420 - 1,2,3 [ CSSD]2013-05-23 16:36:18.350 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x2311420) pid(21547) version 11:1:1:4 [ CSSD]2013-05-23 16:36:18.364 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:36:18.365 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:36:18.368 [1191196992] >TRACE: clssgmDeadProc: proc 0x2311420 [ CSSD]2013-05-23 16:36:18.368 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2311420) con(0x22f2a10) skgpid ospid 21547 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:18.439 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:36:18.439 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x2302db0) skgpid ospid 21542 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:18.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:18.556 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:19.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:19.473 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014679, gin 1, member 0 [ CSSD]2013-05-23 16:36:19.473 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:36:19.473 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:36:19.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:19.480 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:36:19.480 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23291d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:36:19.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:20.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:20.568 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:20.571 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:20.571 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014680, gin 1, member 0 [ CSSD]2013-05-23 16:36:20.571 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:36:20.571 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:36:20.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:20.578 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:36:20.578 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23291d0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:36:21.069 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:21.069 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:21.999 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:22.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:22.574 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:23.270 [1191196992] >TRACE: clssgmAllocProc: (0x22e8310) allocated [ CSSD]2013-05-23 16:36:23.271 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:36:23.271 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8310 - 1,2,3 [ CSSD]2013-05-23 16:36:23.271 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8310) pid(21588) version 11:1:1:4 [ CSSD]2013-05-23 16:36:23.277 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8310), client(1/0x2302db0) [ CSSD]2013-05-23 16:36:23.277 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302db0) [ CSSD]2013-05-23 16:36:23.277 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302db0 with con 0x22e91c0, requested num -1 [ CSSD]2013-05-23 16:36:23.277 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:23.277 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1660 (0x7f431bff2e10) [ CSSD]2013-05-23 16:36:23.277 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2e10 (RPC#1660) tag(67c002c) sent to node 1 [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(067c002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014681, gin 1, member 0 [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2e10 (RPC#1660) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2e10, ret 0, client 0x2302db0 member 0x22e81f0 [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302db0 [ CSSD]2013-05-23 16:36:23.278 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1660 [ CSSD]2013-05-23 16:36:23.284 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302db0) [ CSSD]2013-05-23 16:36:23.284 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302db0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:23.284 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e81f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:23.284 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1661 (0x7f431bff2e90) [ CSSD]2013-05-23 16:36:23.284 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2e90 (RPC#1661) tag(67d002c) sent to node 1 [ CSSD]2013-05-23 16:36:23.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(067d002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:23.285 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:23.285 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2e90 (RPC#1661) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:23.285 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff2e90, ret 0, client 0x2302db0 member 0x22e81f0 [ CSSD]2013-05-23 16:36:23.285 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1661 [ CSSD]2013-05-23 16:36:23.285 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e81f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:36:23.373 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:36:23.373 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1662 (0x7f431bff2f10) [ CSSD]2013-05-23 16:36:23.373 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2f10 (RPC#1662) tag(67e002c) sent to node 1 [ CSSD]2013-05-23 16:36:23.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:23.374 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1662 (0x7f431bff2f10) [ CSSD]2013-05-23 16:36:23.374 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2f10 (RPC#1662) state 6, flags 0x202 [ CSSD]2013-05-23 16:36:23.374 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff2f10, ret 0, client (nil) [ CSSD]2013-05-23 16:36:23.374 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1662 [ CSSD]2013-05-23 16:36:23.487 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8310 [ CSSD]2013-05-23 16:36:23.487 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8310) con(0x22f2a10) skgpid ospid 21588 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:23.938 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2321dd0) [ CSSD]2013-05-23 16:36:23.938 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2321dd0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:36:23.938 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2321dd0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:36:23.938 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 16:36:23.948 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:23.948 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2317d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:36:23.948 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2317d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:36:23.948 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 16:36:23.948 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5060 [ CSSD]2013-05-23 16:36:23.948 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5060) con(0x22e36e0) skgpid 16276 ospid 16276 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:24.011 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:24.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:24.627 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:25.567 [1191196992] >TRACE: clssgmAllocProc: (0x22e8b30) allocated [ CSSD]2013-05-23 16:36:25.568 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:36:25.569 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8b30 - 1,2,3 [ CSSD]2013-05-23 16:36:25.569 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8b30) pid(21618) version 11:1:1:4 [ CSSD]2013-05-23 16:36:25.585 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8b30), client(1/0x2317d30) [ CSSD]2013-05-23 16:36:25.585 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:25.585 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317d30 with con 0x2302db0, requested num -1 [ CSSD]2013-05-23 16:36:25.585 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:25.585 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1663 (0x7f431bff2f90) [ CSSD]2013-05-23 16:36:25.585 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff2f90 (RPC#1663) tag(67f002c) sent to node 1 [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(067f002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014682, gin 1, member 0 [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff2f90 (RPC#1663) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff2f90, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317d30 [ CSSD]2013-05-23 16:36:25.586 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1663 [ CSSD]2013-05-23 16:36:25.592 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:25.592 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:25.592 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e89f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:25.592 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1664 (0x7f431bff3010) [ CSSD]2013-05-23 16:36:25.592 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3010 (RPC#1664) tag(680002c) sent to node 1 [ CSSD]2013-05-23 16:36:25.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0680002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:25.592 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:25.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3010 (RPC#1664) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:25.592 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3010, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:36:25.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1664 [ CSSD]2013-05-23 16:36:25.592 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e89f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:36:25.795 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8b30 [ CSSD]2013-05-23 16:36:25.795 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8b30) con(0x22f2a10) skgpid ospid 21618 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:26.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:26.072 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:26.073 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:26.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:26.652 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:28.028 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:28.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:28.658 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:30.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:30.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:30.670 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:31.075 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:31.075 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:32.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:32.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:32.676 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:33.393 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:36:33.394 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1665 (0x7f431bff3090) [ CSSD]2013-05-23 16:36:33.394 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3090 (RPC#1665) tag(681002c) sent to node 1 [ CSSD]2013-05-23 16:36:33.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:33.394 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1665 (0x7f431bff3090) [ CSSD]2013-05-23 16:36:33.394 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3090 (RPC#1665) state 6, flags 0x202 [ CSSD]2013-05-23 16:36:33.394 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3090, ret 0, client (nil) [ CSSD]2013-05-23 16:36:33.394 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1665 [ CSSD]2013-05-23 16:36:34.042 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:34.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:34.728 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:35.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:35.828 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014683, gin 1, member 0 [ CSSD]2013-05-23 16:36:35.828 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:36:35.828 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:36:35.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:35.835 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:36:35.835 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:36:36.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:36.078 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:36.079 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:36.733 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:36.733 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:38.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:38.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:38.739 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:40.081 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:40.081 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:36:40.104 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:40.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:40.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:40.835 [1191196992] >TRACE: clssgmAllocProc: (0x22e8b30) allocated [ CSSD]2013-05-23 16:36:40.836 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:36:40.837 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8b30 - 1,2,3 [ CSSD]2013-05-23 16:36:40.837 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302db0) proc(0x22e8b30) pid(21773) version 11:1:1:4 [ CSSD]2013-05-23 16:36:40.842 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8b30), client(1/0x2317d30) [ CSSD]2013-05-23 16:36:40.842 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:40.842 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317d30 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 16:36:40.842 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:40.842 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1666 (0x7f431bff3110) [ CSSD]2013-05-23 16:36:40.842 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3110 (RPC#1666) tag(682002c) sent to node 1 [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0682002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014684, gin 1, member 0 [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3110 (RPC#1666) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3110, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317d30 [ CSSD]2013-05-23 16:36:40.843 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1666 [ CSSD]2013-05-23 16:36:40.849 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:40.849 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:40.849 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e89f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:40.849 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1667 (0x7f431bff3190) [ CSSD]2013-05-23 16:36:40.849 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3190 (RPC#1667) tag(683002c) sent to node 1 [ CSSD]2013-05-23 16:36:40.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0683002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:40.850 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:40.850 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3190 (RPC#1667) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:40.850 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3190, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:36:40.850 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1667 [ CSSD]2013-05-23 16:36:40.850 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e89f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:36:41.052 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8b30 [ CSSD]2013-05-23 16:36:41.052 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8b30) con(0x2302db0) skgpid ospid 21773 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:42.109 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:42.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:42.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:43.433 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:36:43.434 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1668 (0x7f431bff3210) [ CSSD]2013-05-23 16:36:43.434 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3210 (RPC#1668) tag(684002c) sent to node 1 [ CSSD]2013-05-23 16:36:43.435 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:43.435 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1668 (0x7f431bff3210) [ CSSD]2013-05-23 16:36:43.435 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3210 (RPC#1668) state 6, flags 0x202 [ CSSD]2013-05-23 16:36:43.435 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3210, ret 0, client (nil) [ CSSD]2013-05-23 16:36:43.435 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1668 [ CSSD]2013-05-23 16:36:44.122 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:44.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:44.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:45.084 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:45.085 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:46.127 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:46.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:46.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:48.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:48.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:48.798 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:50.088 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:50.088 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:50.185 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:50.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:50.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:51.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:51.088 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014685, gin 1, member 0 [ CSSD]2013-05-23 16:36:51.088 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:36:51.088 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:36:51.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:51.095 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:36:51.095 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:36:52.210 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:52.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:52.855 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:53.453 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:36:53.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1669 (0x7f431bff3290) [ CSSD]2013-05-23 16:36:53.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3290 (RPC#1669) tag(685002c) sent to node 1 [ CSSD]2013-05-23 16:36:53.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:53.454 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1669 (0x7f431bff3290) [ CSSD]2013-05-23 16:36:53.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3290 (RPC#1669) state 6, flags 0x202 [ CSSD]2013-05-23 16:36:53.454 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3290, ret 0, client (nil) [ CSSD]2013-05-23 16:36:53.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1669 [ CSSD]2013-05-23 16:36:54.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:54.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:54.867 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:55.091 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:55.091 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:36:56.083 [1191196992] >TRACE: clssgmAllocProc: (0x22e8b30) allocated [ CSSD]2013-05-23 16:36:56.084 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:36:56.085 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8b30 - 1,2,3 [ CSSD]2013-05-23 16:36:56.085 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302db0) proc(0x22e8b30) pid(21978) version 11:1:1:4 [ CSSD]2013-05-23 16:36:56.090 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8b30), client(1/0x2317d30) [ CSSD]2013-05-23 16:36:56.090 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:56.090 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317d30 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 16:36:56.090 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:56.090 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1670 (0x7f431bff3310) [ CSSD]2013-05-23 16:36:56.090 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3310 (RPC#1670) tag(686002c) sent to node 1 [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0686002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014686, gin 1, member 0 [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3310 (RPC#1670) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3310, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317d30 [ CSSD]2013-05-23 16:36:56.091 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1670 [ CSSD]2013-05-23 16:36:56.097 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:36:56.097 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:56.097 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e89f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:36:56.097 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1671 (0x7f431bff3390) [ CSSD]2013-05-23 16:36:56.097 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3390 (RPC#1671) tag(687002c) sent to node 1 [ CSSD]2013-05-23 16:36:56.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0687002c) incarnation(174631051) [ CSSD]2013-05-23 16:36:56.098 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:36:56.098 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3390 (RPC#1671) state 6, flags 0x100 [ CSSD]2013-05-23 16:36:56.098 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3390, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:36:56.098 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1671 [ CSSD]2013-05-23 16:36:56.098 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e89f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:36:56.226 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:56.300 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8b30 [ CSSD]2013-05-23 16:36:56.300 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8b30) con(0x2302db0) skgpid ospid 21978 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:36:56.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:56.873 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:58.319 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:36:58.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:36:58.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:36:59.093 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:36:59.093 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:37:00.344 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:00.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:00.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:02.368 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:02.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:02.956 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:03.483 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:37:03.484 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1672 (0x7f431bff3410) [ CSSD]2013-05-23 16:37:03.484 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3410 (RPC#1672) tag(688002c) sent to node 1 [ CSSD]2013-05-23 16:37:03.484 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:03.484 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1672 (0x7f431bff3410) [ CSSD]2013-05-23 16:37:03.484 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3410 (RPC#1672) state 6, flags 0x202 [ CSSD]2013-05-23 16:37:03.484 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3410, ret 0, client (nil) [ CSSD]2013-05-23 16:37:03.485 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1672 [ CSSD]2013-05-23 16:37:04.096 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:04.096 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:04.402 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:04.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:04.968 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:06.342 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:06.342 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014687, gin 1, member 0 [ CSSD]2013-05-23 16:37:06.342 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:06.342 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:06.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:06.349 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:06.349 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:06.426 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:06.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:06.974 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:08.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:09.027 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:09.027 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:09.099 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:09.099 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:10.543 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:11.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:11.032 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:11.342 [1191196992] >TRACE: clssgmAllocProc: (0x22e8b30) allocated [ CSSD]2013-05-23 16:37:11.343 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:37:11.344 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8b30 - 1,2,3 [ CSSD]2013-05-23 16:37:11.344 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302db0) proc(0x22e8b30) pid(22132) version 11:1:1:4 [ CSSD]2013-05-23 16:37:11.350 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8b30), client(1/0x2317d30) [ CSSD]2013-05-23 16:37:11.350 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:37:11.350 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317d30 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 16:37:11.350 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:11.350 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1673 (0x7f431bff3490) [ CSSD]2013-05-23 16:37:11.350 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3490 (RPC#1673) tag(689002c) sent to node 1 [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0689002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014688, gin 1, member 0 [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e40b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3490 (RPC#1673) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3490, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317d30 [ CSSD]2013-05-23 16:37:11.351 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1673 [ CSSD]2013-05-23 16:37:11.357 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317d30) [ CSSD]2013-05-23 16:37:11.357 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:11.357 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e89f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:11.357 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1674 (0x7f431bff3510) [ CSSD]2013-05-23 16:37:11.358 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3510 (RPC#1674) tag(68a002c) sent to node 1 [ CSSD]2013-05-23 16:37:11.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(068a002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:11.358 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:11.358 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3510 (RPC#1674) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:11.358 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3510, ret 0, client 0x2317d30 member 0x22e89f0 [ CSSD]2013-05-23 16:37:11.358 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1674 [ CSSD]2013-05-23 16:37:11.358 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e89f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:37:11.560 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8b30 [ CSSD]2013-05-23 16:37:11.560 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8b30) con(0x2302db0) skgpid ospid 22132 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:12.548 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:12.569 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:12.569 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014689, gin 1, member 0 [ CSSD]2013-05-23 16:37:12.570 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:12.570 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:12.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:12.575 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:12.575 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:13.037 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:13.038 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:13.513 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:37:13.514 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1675 (0x7f431bff3590) [ CSSD]2013-05-23 16:37:13.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3590 (RPC#1675) tag(68b002c) sent to node 1 [ CSSD]2013-05-23 16:37:13.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:13.514 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1675 (0x7f431bff3590) [ CSSD]2013-05-23 16:37:13.514 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3590 (RPC#1675) state 6, flags 0x202 [ CSSD]2013-05-23 16:37:13.514 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3590, ret 0, client (nil) [ CSSD]2013-05-23 16:37:13.515 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1675 [ CSSD]2013-05-23 16:37:14.102 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:14.102 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:14.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:15.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:15.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:16.668 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:16.722 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 16:37:16.723 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:37:16.723 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 16:37:16.723 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(22405) version 11:1:1:4 [ CSSD]2013-05-23 16:37:16.842 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 16:37:16.843 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:37:16.843 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 16:37:16.843 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c13ea10) pid(22415) version 11:1:1:4 [ CSSD]2013-05-23 16:37:16.859 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:37:16.860 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:37:16.863 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 16:37:16.863 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c11ab50) skgpid ospid 22415 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:16.941 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(70/0x7f431c10a560) [ CSSD]2013-05-23 16:37:16.941 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 70 (0x7f431c10a560) [ CSSD]2013-05-23 16:37:16.941 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c10a560 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:37:16.941 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:37:16.941 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1676 (0x7f431bff3610) [ CSSD]2013-05-23 16:37:16.942 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3610 (RPC#1676) tag(68c002c) sent to node 1 [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(068c002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014690, gin 1, member 0 [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c13edd0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3610 (RPC#1676) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3610, ret 0, client 0x7f431c10a560 member 0x7f431c12f520 [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c10a560 [ CSSD]2013-05-23 16:37:16.942 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1676 [ CSSD]2013-05-23 16:37:16.948 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 70 (0x7f431c10a560) [ CSSD]2013-05-23 16:37:16.948 [1191196992] >TRACE: clssgmExitGrock: client 70 (0x7f431c10a560), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:37:16.948 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:37:16.948 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1677 (0x7f431bff3690) [ CSSD]2013-05-23 16:37:16.948 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3690 (RPC#1677) tag(68d002c) sent to node 1 [ CSSD]2013-05-23 16:37:16.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(068d002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:16.948 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:37:16.948 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3690 (RPC#1677) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:16.948 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3690, ret 0, client 0x7f431c10a560 member 0x7f431c12f520 [ CSSD]2013-05-23 16:37:16.948 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1677 [ CSSD]2013-05-23 16:37:16.949 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:37:16.949 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 16:37:16.949 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 22405 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:17.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:17.073 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:18.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:19.105 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:19.105 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:19.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:19.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:19.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:19.728 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014691, gin 1, member 0 [ CSSD]2013-05-23 16:37:19.728 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:19.728 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:19.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:19.735 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:19.735 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:20.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:21.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:21.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:21.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:21.598 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014692, gin 1, member 0 [ CSSD]2013-05-23 16:37:21.598 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:21.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:21.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:21.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:21.605 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:22.719 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:23.107 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:23.107 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:37:23.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:23.138 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:23.530 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:37:23.531 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:37:23.531 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:37:23.531 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c10a190) pid(22467) version 11:1:1:4 [ CSSD]2013-05-23 16:37:23.533 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:37:23.533 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1678 (0x7f431bff3710) [ CSSD]2013-05-23 16:37:23.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3710 (RPC#1678) tag(68e002c) sent to node 1 [ CSSD]2013-05-23 16:37:23.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:23.534 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1678 (0x7f431bff3710) [ CSSD]2013-05-23 16:37:23.534 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3710 (RPC#1678) state 6, flags 0x202 [ CSSD]2013-05-23 16:37:23.534 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3710, ret 0, client (nil) [ CSSD]2013-05-23 16:37:23.534 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1678 [ CSSD]2013-05-23 16:37:23.537 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c10a190), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:37:23.537 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:37:23.537 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:37:23.537 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:23.537 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1679 (0x7f431bff3790) [ CSSD]2013-05-23 16:37:23.537 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3790 (RPC#1679) tag(68f002c) sent to node 1 [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(068f002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014693, gin 1, member 0 [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3790 (RPC#1679) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3790, ret 0, client 0x7f431c109f80 member 0x7f431c048500 [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:37:23.538 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1679 [ CSSD]2013-05-23 16:37:23.544 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:37:23.544 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:23.544 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:23.544 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1680 (0x7f431bff3810) [ CSSD]2013-05-23 16:37:23.544 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3810 (RPC#1680) tag(690002c) sent to node 1 [ CSSD]2013-05-23 16:37:23.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0690002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:23.545 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:23.545 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3810 (RPC#1680) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:23.545 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3810, ret 0, client 0x7f431c109f80 member 0x7f431c048500 [ CSSD]2013-05-23 16:37:23.545 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1680 [ CSSD]2013-05-23 16:37:23.545 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:37:23.747 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:37:23.747 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e6720) skgpid ospid 22467 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:24.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:25.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:25.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:26.588 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 16:37:26.589 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:37:26.590 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 16:37:26.590 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c11fd30) pid(22498) version 11:1:1:4 [ CSSD]2013-05-23 16:37:26.595 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:37:26.595 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:37:26.595 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:37:26.595 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:26.595 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1681 (0x7f431bff3890) [ CSSD]2013-05-23 16:37:26.595 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3890 (RPC#1681) tag(691002c) sent to node 1 [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0691002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014694, gin 1, member 0 [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3890 (RPC#1681) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3890, ret 0, client 0x7f431c109f80 member 0x7f431c048500 [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:37:26.596 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1681 [ CSSD]2013-05-23 16:37:26.603 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:37:26.603 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:26.603 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:26.603 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1682 (0x7f431bff3910) [ CSSD]2013-05-23 16:37:26.603 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3910 (RPC#1682) tag(692002c) sent to node 1 [ CSSD]2013-05-23 16:37:26.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0692002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:26.603 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:26.603 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3910 (RPC#1682) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:26.603 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3910, ret 0, client 0x7f431c109f80 member 0x7f431c048500 [ CSSD]2013-05-23 16:37:26.604 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1682 [ CSSD]2013-05-23 16:37:26.604 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:37:26.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:26.806 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 16:37:26.806 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e6720) skgpid ospid 22498 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:27.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:27.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:28.110 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:28.110 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:28.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:29.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:29.177 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:30.746 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:31.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:31.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:32.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:33.113 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:33.113 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:33.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:33.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:33.553 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:37:33.553 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1683 (0x7f431bff3990) [ CSSD]2013-05-23 16:37:33.553 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3990 (RPC#1683) tag(693002c) sent to node 1 [ CSSD]2013-05-23 16:37:33.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:33.554 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1683 (0x7f431bff3990) [ CSSD]2013-05-23 16:37:33.554 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3990 (RPC#1683) state 6, flags 0x202 [ CSSD]2013-05-23 16:37:33.554 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3990, ret 0, client (nil) [ CSSD]2013-05-23 16:37:33.554 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1683 [ CSSD]2013-05-23 16:37:34.805 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:35.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:35.240 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:36.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:36.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:36.848 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014695, gin 1, member 0 [ CSSD]2013-05-23 16:37:36.848 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:36.848 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:36.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:36.855 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:36.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:37.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:37.246 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:38.116 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:38.117 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:38.822 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:39.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:39.259 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:40.827 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:41.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:41.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:41.847 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:37:41.848 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:37:41.848 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:37:41.848 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c10a190) pid(22637) version 11:1:1:4 [ CSSD]2013-05-23 16:37:41.853 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c10a190), client(1/0x7f431c109f80) [ CSSD]2013-05-23 16:37:41.853 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:37:41.853 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:37:41.853 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:41.853 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1684 (0x7f431bff3a10) [ CSSD]2013-05-23 16:37:41.853 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3a10 (RPC#1684) tag(694002c) sent to node 1 [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0694002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014696, gin 1, member 0 [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3a10 (RPC#1684) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3a10, ret 0, client 0x7f431c109f80 member 0x7f431c12f520 [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 16:37:41.854 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1684 [ CSSD]2013-05-23 16:37:41.860 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 16:37:41.860 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:41.860 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:41.860 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1685 (0x7f431bff3a90) [ CSSD]2013-05-23 16:37:41.860 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3a90 (RPC#1685) tag(695002c) sent to node 1 [ CSSD]2013-05-23 16:37:41.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0695002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:41.860 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:41.860 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3a90 (RPC#1685) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:41.860 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3a90, ret 0, client 0x7f431c109f80 member 0x7f431c12f520 [ CSSD]2013-05-23 16:37:41.861 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1685 [ CSSD]2013-05-23 16:37:41.861 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:37:42.063 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:37:42.063 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e5200) skgpid ospid 22637 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:42.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:42.094 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014697, gin 1, member 0 [ CSSD]2013-05-23 16:37:42.094 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:42.094 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:42.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:42.101 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:42.101 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:42.850 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:43.120 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:43.120 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:43.269 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:43.269 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:43.574 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:37:43.574 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1686 (0x7f431bff3b10) [ CSSD]2013-05-23 16:37:43.574 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3b10 (RPC#1686) tag(696002c) sent to node 1 [ CSSD]2013-05-23 16:37:43.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:43.575 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1686 (0x7f431bff3b10) [ CSSD]2013-05-23 16:37:43.575 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3b10 (RPC#1686) state 6, flags 0x202 [ CSSD]2013-05-23 16:37:43.575 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3b10, ret 0, client (nil) [ CSSD]2013-05-23 16:37:43.575 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1686 [ CSSD]2013-05-23 16:37:44.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:45.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:45.281 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:46.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:47.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:47.287 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:48.123 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:48.123 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:48.912 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:49.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:49.338 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:49.348 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2317b60) [ CSSD]2013-05-23 16:37:49.348 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2317b60), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:37:49.348 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2317b60), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:37:49.348 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:37:49.358 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x230f280) [ CSSD]2013-05-23 16:37:49.358 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x230f280), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:37:49.358 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x230f280), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:37:49.358 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:37:49.358 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 16:37:49.358 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c12f0c0) skgpid 17399 ospid 17399 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:50.917 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:51.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:51.363 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:52.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:52.104 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014698, gin 1, member 0 [ CSSD]2013-05-23 16:37:52.104 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:37:52.104 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:37:52.111 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:52.111 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:37:52.111 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9100) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:37:53.010 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:53.126 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:53.126 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:53.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:53.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:53.594 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:37:53.595 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1687 (0x7f431bff3b90) [ CSSD]2013-05-23 16:37:53.595 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3b90 (RPC#1687) tag(697002c) sent to node 1 [ CSSD]2013-05-23 16:37:53.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:53.595 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1687 (0x7f431bff3b90) [ CSSD]2013-05-23 16:37:53.595 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3b90 (RPC#1687) state 6, flags 0x202 [ CSSD]2013-05-23 16:37:53.595 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3b90, ret 0, client (nil) [ CSSD]2013-05-23 16:37:53.595 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1687 [ CSSD]2013-05-23 16:37:55.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:55.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:55.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:57.021 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:57.094 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135050) allocated [ CSSD]2013-05-23 16:37:57.095 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:37:57.095 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135050 - 1,2,3 [ CSSD]2013-05-23 16:37:57.095 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c135050) pid(22868) version 11:1:1:4 [ CSSD]2013-05-23 16:37:57.101 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c135050), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 16:37:57.101 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:37:57.101 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:37:57.101 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:57.101 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1688 (0x7f431bff3c10) [ CSSD]2013-05-23 16:37:57.102 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3c10 (RPC#1688) tag(698002c) sent to node 1 [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0698002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014699, gin 1, member 0 [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3c10 (RPC#1688) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3c10, ret 0, client 0x7f431c0e5910 member 0x7f431c10a490 [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 16:37:57.102 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1688 [ CSSD]2013-05-23 16:37:57.109 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 16:37:57.109 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:57.109 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:37:57.109 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1689 (0x7f431bff3c90) [ CSSD]2013-05-23 16:37:57.109 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3c90 (RPC#1689) tag(699002c) sent to node 1 [ CSSD]2013-05-23 16:37:57.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0699002c) incarnation(174631051) [ CSSD]2013-05-23 16:37:57.109 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:37:57.109 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3c90 (RPC#1689) state 6, flags 0x100 [ CSSD]2013-05-23 16:37:57.109 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3c90, ret 0, client 0x7f431c0e5910 member 0x7f431c10a490 [ CSSD]2013-05-23 16:37:57.110 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1689 [ CSSD]2013-05-23 16:37:57.110 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:37:57.312 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135050 [ CSSD]2013-05-23 16:37:57.312 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135050) con(0x7f431c12f0c0) skgpid ospid 22868 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:37:57.387 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:57.387 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:37:58.129 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:37:58.129 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:37:59.033 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:37:59.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:37:59.440 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:01.038 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:01.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:01.465 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:03.050 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:03.132 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:03.132 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:03.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:03.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:03.614 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:38:03.614 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1690 (0x7f431bff3d10) [ CSSD]2013-05-23 16:38:03.614 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3d10 (RPC#1690) tag(69a002c) sent to node 1 [ CSSD]2013-05-23 16:38:03.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:03.615 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1690 (0x7f431bff3d10) [ CSSD]2013-05-23 16:38:03.615 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3d10 (RPC#1690) state 6, flags 0x202 [ CSSD]2013-05-23 16:38:03.615 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3d10, ret 0, client (nil) [ CSSD]2013-05-23 16:38:03.615 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1690 [ CSSD]2013-05-23 16:38:05.056 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:05.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:05.481 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:07.061 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:07.135 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:07.135 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:38:07.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:07.355 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014700, gin 1, member 0 [ CSSD]2013-05-23 16:38:07.355 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:38:07.355 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:38:07.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:07.362 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:38:07.362 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2317b60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:38:07.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:07.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:09.113 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:09.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:09.540 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:11.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:11.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:11.566 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:12.138 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:12.138 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:12.343 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 16:38:12.344 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:38:12.344 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 16:38:12.344 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c108da0) pid(23034) version 11:1:1:4 [ CSSD]2013-05-23 16:38:12.350 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c108da0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:38:12.350 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:38:12.350 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:38:12.350 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:12.350 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1691 (0x7f431bff3d90) [ CSSD]2013-05-23 16:38:12.350 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3d90 (RPC#1691) tag(69b002c) sent to node 1 [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(069b002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014701, gin 1, member 0 [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3d90 (RPC#1691) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3d90, ret 0, client 0x7f431c0cf720 member 0x7f431c0e6d60 [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:38:12.351 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1691 [ CSSD]2013-05-23 16:38:12.358 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:38:12.358 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:12.358 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:12.358 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1692 (0x7f431bff3e10) [ CSSD]2013-05-23 16:38:12.358 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3e10 (RPC#1692) tag(69c002c) sent to node 1 [ CSSD]2013-05-23 16:38:12.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(069c002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:12.358 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:12.358 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3e10 (RPC#1692) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:12.358 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff3e10, ret 0, client 0x7f431c0cf720 member 0x7f431c0e6d60 [ CSSD]2013-05-23 16:38:12.359 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1692 [ CSSD]2013-05-23 16:38:12.359 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:38:12.561 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 16:38:12.561 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c11ab50) skgpid ospid 23034 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:38:13.131 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:13.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:13.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:13.644 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:38:13.644 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1693 (0x7f431bff3e90) [ CSSD]2013-05-23 16:38:13.644 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3e90 (RPC#1693) tag(69d002c) sent to node 1 [ CSSD]2013-05-23 16:38:13.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:13.645 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1693 (0x7f431bff3e90) [ CSSD]2013-05-23 16:38:13.645 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3e90 (RPC#1693) state 6, flags 0x202 [ CSSD]2013-05-23 16:38:13.645 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3e90, ret 0, client (nil) [ CSSD]2013-05-23 16:38:13.645 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1693 [ CSSD]2013-05-23 16:38:15.136 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:15.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:15.683 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:16.621 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 16:38:16.621 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:38:16.622 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 16:38:16.622 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c10a490) pid(22790/22790) version 11:1:1:4 [ CSSD]2013-05-23 16:38:16.637 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a490), client(1/0x22f1d20) [ CSSD]2013-05-23 16:38:16.637 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1d20) [ CSSD]2013-05-23 16:38:16.637 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:38:16.637 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:38:17.097 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a490), client(2/0x22e91d0) [ CSSD]2013-05-23 16:38:17.097 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22e91d0) [ CSSD]2013-05-23 16:38:17.097 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:38:17.097 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:38:17.141 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:17.141 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:17.141 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:17.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:17.689 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:19.152 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:19.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:19.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:19.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:19.996 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014702, gin 1, member 0 [ CSSD]2013-05-23 16:38:19.996 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:38:19.996 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:38:20.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:20.003 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:38:20.003 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:38:21.158 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:21.766 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:21.766 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:22.144 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:22.144 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:22.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:22.606 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014703, gin 1, member 0 [ CSSD]2013-05-23 16:38:22.606 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:38:22.606 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:38:22.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:22.612 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:38:22.612 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:38:23.211 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:23.674 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:38:23.674 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1694 (0x7f431bff3f10) [ CSSD]2013-05-23 16:38:23.674 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3f10 (RPC#1694) tag(69e002c) sent to node 1 [ CSSD]2013-05-23 16:38:23.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:23.675 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1694 (0x7f431bff3f10) [ CSSD]2013-05-23 16:38:23.675 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3f10 (RPC#1694) state 6, flags 0x202 [ CSSD]2013-05-23 16:38:23.675 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff3f10, ret 0, client (nil) [ CSSD]2013-05-23 16:38:23.675 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1694 [ CSSD]2013-05-23 16:38:23.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:23.772 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:23.793 [1191196992] >TRACE: clssgmAllocProc: (0x22e8a60) allocated [ CSSD]2013-05-23 16:38:23.794 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:38:23.794 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8a60 - 1,2,3 [ CSSD]2013-05-23 16:38:23.794 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22e8a60) pid(23151) version 11:1:1:4 [ CSSD]2013-05-23 16:38:23.800 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8a60), client(1/0x2317c30) [ CSSD]2013-05-23 16:38:23.800 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317c30) [ CSSD]2013-05-23 16:38:23.800 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317c30 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 16:38:23.801 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:23.801 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1695 (0x7f431bff3f90) [ CSSD]2013-05-23 16:38:23.801 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff3f90 (RPC#1695) tag(69f002c) sent to node 1 [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(069f002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014704, gin 1, member 0 [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff3f90 (RPC#1695) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff3f90, ret 0, client 0x2317c30 member 0x22e8940 [ CSSD]2013-05-23 16:38:23.801 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317c30 [ CSSD]2013-05-23 16:38:23.802 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1695 [ CSSD]2013-05-23 16:38:23.808 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317c30) [ CSSD]2013-05-23 16:38:23.808 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317c30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:23.808 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:23.808 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1696 (0x7f431bff4010) [ CSSD]2013-05-23 16:38:23.808 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4010 (RPC#1696) tag(6a0002c) sent to node 1 [ CSSD]2013-05-23 16:38:23.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06a0002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:23.808 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:23.808 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4010 (RPC#1696) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:23.808 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4010, ret 0, client 0x2317c30 member 0x22e8940 [ CSSD]2013-05-23 16:38:23.809 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1696 [ CSSD]2013-05-23 16:38:23.809 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:38:24.011 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8a60 [ CSSD]2013-05-23 16:38:24.011 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8a60) con(0x230f280) skgpid ospid 23151 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:38:25.216 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:25.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:25.783 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:26.146 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:26.146 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:38:27.222 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:27.603 [1191196992] >TRACE: clssgmAllocProc: (0x22e8a60) allocated [ CSSD]2013-05-23 16:38:27.603 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:38:27.604 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8a60 - 1,2,3 [ CSSD]2013-05-23 16:38:27.604 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8a60) pid(23185) version 11:1:1:4 [ CSSD]2013-05-23 16:38:27.610 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8a60), client(1/0x2317c30) [ CSSD]2013-05-23 16:38:27.610 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317c30) [ CSSD]2013-05-23 16:38:27.610 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317c30 with con 0x22e8ca0, requested num -1 [ CSSD]2013-05-23 16:38:27.610 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:27.610 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1697 (0x7f431bff4090) [ CSSD]2013-05-23 16:38:27.611 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4090 (RPC#1697) tag(6a1002c) sent to node 1 [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06a1002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014705, gin 1, member 0 [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3ff0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4090 (RPC#1697) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4090, ret 0, client 0x2317c30 member 0x22e8940 [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317c30 [ CSSD]2013-05-23 16:38:27.611 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1697 [ CSSD]2013-05-23 16:38:27.617 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317c30) [ CSSD]2013-05-23 16:38:27.617 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317c30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:27.617 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:27.617 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1698 (0x7f431bff4110) [ CSSD]2013-05-23 16:38:27.618 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4110 (RPC#1698) tag(6a2002c) sent to node 1 [ CSSD]2013-05-23 16:38:27.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06a2002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:27.618 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:27.618 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4110 (RPC#1698) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:27.618 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4110, ret 0, client 0x2317c30 member 0x22e8940 [ CSSD]2013-05-23 16:38:27.618 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1698 [ CSSD]2013-05-23 16:38:27.618 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:38:27.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:27.789 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:27.821 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8a60 [ CSSD]2013-05-23 16:38:27.821 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8a60) con(0x22f2a10) skgpid ospid 23185 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:38:29.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:29.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:29.841 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:31.149 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:31.149 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:31.278 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:31.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:31.847 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:33.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:33.704 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:38:33.704 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1699 (0x7f431bff4190) [ CSSD]2013-05-23 16:38:33.704 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4190 (RPC#1699) tag(6a3002c) sent to node 1 [ CSSD]2013-05-23 16:38:33.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:33.705 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1699 (0x7f431bff4190) [ CSSD]2013-05-23 16:38:33.705 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4190 (RPC#1699) state 6, flags 0x202 [ CSSD]2013-05-23 16:38:33.705 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4190, ret 0, client (nil) [ CSSD]2013-05-23 16:38:33.705 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1699 [ CSSD]2013-05-23 16:38:33.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:33.873 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:35.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:35.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:35.883 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:36.152 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:36.153 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:37.324 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:37.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:37.864 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014706, gin 1, member 0 [ CSSD]2013-05-23 16:38:37.864 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:38:37.864 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:38:37.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:37.871 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:38:37.871 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:38:37.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:37.889 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:39.335 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:39.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:39.354 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014707, gin 1, member 0 [ CSSD]2013-05-23 16:38:39.354 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:38:39.355 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:38:39.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:39.361 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:38:39.361 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:38:39.497 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:38:39.497 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:38:39.499 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:38:39.499 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:38:39.501 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:38:39.502 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:38:39.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:39.942 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:41.155 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:41.156 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:41.359 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:41.947 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:41.947 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:42.852 [1191196992] >TRACE: clssgmAllocProc: (0x22e8a60) allocated [ CSSD]2013-05-23 16:38:42.853 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:38:42.854 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8a60 - 1,2,3 [ CSSD]2013-05-23 16:38:42.854 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x22e8a60) pid(23353) version 11:1:1:4 [ CSSD]2013-05-23 16:38:42.859 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e8a60), client(1/0x2317c30) [ CSSD]2013-05-23 16:38:42.860 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317c30) [ CSSD]2013-05-23 16:38:42.860 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317c30 with con 0x230f280, requested num -1 [ CSSD]2013-05-23 16:38:42.860 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:42.860 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1700 (0x7f431bff4210) [ CSSD]2013-05-23 16:38:42.860 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4210 (RPC#1700) tag(6a4002c) sent to node 1 [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06a4002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014708, gin 1, member 0 [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4210 (RPC#1700) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4210, ret 0, client 0x2317c30 member 0x22e8940 [ CSSD]2013-05-23 16:38:42.860 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317c30 [ CSSD]2013-05-23 16:38:42.861 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1700 [ CSSD]2013-05-23 16:38:42.867 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317c30) [ CSSD]2013-05-23 16:38:42.867 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317c30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:42.867 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e8940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:42.867 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1701 (0x7f431bff4290) [ CSSD]2013-05-23 16:38:42.867 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4290 (RPC#1701) tag(6a5002c) sent to node 1 [ CSSD]2013-05-23 16:38:42.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06a5002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:42.868 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:42.868 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4290 (RPC#1701) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:42.868 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4290, ret 0, client 0x2317c30 member 0x22e8940 [ CSSD]2013-05-23 16:38:42.868 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1701 [ CSSD]2013-05-23 16:38:42.868 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e8940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:38:43.070 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8a60 [ CSSD]2013-05-23 16:38:43.070 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8a60) con(0x22f2a10) skgpid ospid 23353 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:38:43.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:43.725 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:38:43.725 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1702 (0x7f431bff4310) [ CSSD]2013-05-23 16:38:43.725 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4310 (RPC#1702) tag(6a6002c) sent to node 1 [ CSSD]2013-05-23 16:38:43.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:43.726 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1702 (0x7f431bff4310) [ CSSD]2013-05-23 16:38:43.726 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4310 (RPC#1702) state 6, flags 0x202 [ CSSD]2013-05-23 16:38:43.726 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4310, ret 0, client (nil) [ CSSD]2013-05-23 16:38:43.726 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1702 [ CSSD]2013-05-23 16:38:43.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:43.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:45.418 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:45.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:45.964 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:46.159 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:46.159 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:47.423 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:47.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:47.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:49.435 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:49.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:49.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:51.162 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:51.162 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:51.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:52.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:52.017 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:53.114 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:53.114 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014709, gin 1, member 0 [ CSSD]2013-05-23 16:38:53.114 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2328ed0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:38:53.114 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:38:53.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:53.120 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:38:53.120 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ee070) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:38:53.452 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:53.744 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:38:53.744 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1703 (0x7f431bff4390) [ CSSD]2013-05-23 16:38:53.744 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4390 (RPC#1703) tag(6a7002c) sent to node 1 [ CSSD]2013-05-23 16:38:53.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:53.745 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1703 (0x7f431bff4390) [ CSSD]2013-05-23 16:38:53.745 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4390 (RPC#1703) state 6, flags 0x202 [ CSSD]2013-05-23 16:38:53.745 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4390, ret 0, client (nil) [ CSSD]2013-05-23 16:38:53.745 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1703 [ CSSD]2013-05-23 16:38:53.985 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f1bb0) [ CSSD]2013-05-23 16:38:53.985 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f1bb0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:38:53.985 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f1bb0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:38:53.985 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:38:53.990 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2317ea0) [ CSSD]2013-05-23 16:38:53.990 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2317ea0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:38:53.990 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2317ea0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:38:53.990 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:38:53.990 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f7c20 [ CSSD]2013-05-23 16:38:53.990 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f7c20) con(0x22f4b70) skgpid 18164 ospid 18164 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:38:54.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:54.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:55.497 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:56.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:56.055 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:56.165 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:38:56.165 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:38:57.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:38:58.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:38:58.061 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:38:58.102 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 16:38:58.103 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:38:58.104 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 16:38:58.104 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(23505) version 11:1:1:4 [ CSSD]2013-05-23 16:38:58.109 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:38:58.109 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:38:58.109 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:38:58.109 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:58.109 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1704 (0x7f431bff4410) [ CSSD]2013-05-23 16:38:58.110 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4410 (RPC#1704) tag(6a8002c) sent to node 1 [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06a8002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014710, gin 1, member 0 [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4410 (RPC#1704) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4410, ret 0, client 0x7f431c135120 member 0x7f431c12f520 [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:38:58.110 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1704 [ CSSD]2013-05-23 16:38:58.117 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:38:58.117 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:58.117 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:38:58.117 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1705 (0x7f431bff4490) [ CSSD]2013-05-23 16:38:58.117 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4490 (RPC#1705) tag(6a9002c) sent to node 1 [ CSSD]2013-05-23 16:38:58.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06a9002c) incarnation(174631051) [ CSSD]2013-05-23 16:38:58.117 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:38:58.117 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4490 (RPC#1705) state 6, flags 0x100 [ CSSD]2013-05-23 16:38:58.117 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4490, ret 0, client 0x7f431c135120 member 0x7f431c12f520 [ CSSD]2013-05-23 16:38:58.118 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1705 [ CSSD]2013-05-23 16:38:58.118 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:38:58.320 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 16:38:58.320 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 23505 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:38:59.516 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:00.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:00.071 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:01.168 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:01.169 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:01.540 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:02.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:02.096 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:03.551 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:03.765 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:39:03.765 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1706 (0x7f431bff4510) [ CSSD]2013-05-23 16:39:03.765 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4510 (RPC#1706) tag(6aa002c) sent to node 1 [ CSSD]2013-05-23 16:39:03.766 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:03.766 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1706 (0x7f431bff4510) [ CSSD]2013-05-23 16:39:03.766 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4510 (RPC#1706) state 6, flags 0x202 [ CSSD]2013-05-23 16:39:03.766 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4510, ret 0, client (nil) [ CSSD]2013-05-23 16:39:03.766 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1706 [ CSSD]2013-05-23 16:39:04.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:04.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:05.557 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:06.172 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:06.172 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:06.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:06.194 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:07.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:08.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:08.200 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:08.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:08.360 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014711, gin 1, member 0 [ CSSD]2013-05-23 16:39:08.360 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:39:08.360 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:39:08.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:08.365 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:39:08.365 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:39:09.585 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:10.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:10.252 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:11.062 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:39:11.063 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:39:11.063 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:39:11.063 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c10a190) pid(23478/23478) version 11:1:1:4 [ CSSD]2013-05-23 16:39:11.080 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a190), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:39:11.080 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:39:11.080 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:39:11.080 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:39:11.175 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:11.175 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:11.590 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:12.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:12.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:13.352 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:39:13.353 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:39:13.353 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:39:13.353 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c1246d0) pid(23654) version 11:1:1:4 [ CSSD]2013-05-23 16:39:13.359 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1246d0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:39:13.359 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:39:13.359 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5900, requested num -1 [ CSSD]2013-05-23 16:39:13.359 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:13.359 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1707 (0x7f431bff4590) [ CSSD]2013-05-23 16:39:13.359 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4590 (RPC#1707) tag(6ab002c) sent to node 1 [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06ab002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014712, gin 1, member 0 [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4590 (RPC#1707) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4590, ret 0, client 0x7f431c0e6e30 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:39:13.360 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1707 [ CSSD]2013-05-23 16:39:13.367 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:39:13.367 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:13.367 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1215d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:13.367 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1708 (0x7f431bff4610) [ CSSD]2013-05-23 16:39:13.367 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4610 (RPC#1708) tag(6ac002c) sent to node 1 [ CSSD]2013-05-23 16:39:13.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06ac002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:13.368 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:13.368 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4610 (RPC#1708) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:13.368 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4610, ret 0, client 0x7f431c0e6e30 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:13.368 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1708 [ CSSD]2013-05-23 16:39:13.368 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1215d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:39:13.570 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:39:13.570 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c109f70) skgpid ospid 23654 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:39:13.624 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:13.785 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:39:13.785 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1709 (0x7f431bff4690) [ CSSD]2013-05-23 16:39:13.785 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4690 (RPC#1709) tag(6ad002c) sent to node 1 [ CSSD]2013-05-23 16:39:13.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:13.786 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1709 (0x7f431bff4690) [ CSSD]2013-05-23 16:39:13.786 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4690 (RPC#1709) state 6, flags 0x202 [ CSSD]2013-05-23 16:39:13.786 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4690, ret 0, client (nil) [ CSSD]2013-05-23 16:39:13.786 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1709 [ CSSD]2013-05-23 16:39:14.290 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:14.290 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:15.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:16.177 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:16.177 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:16.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:16.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:17.660 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:18.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:18.300 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:19.665 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:20.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:20.259 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014713, gin 1, member 0 [ CSSD]2013-05-23 16:39:20.259 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:39:20.259 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:39:20.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:20.266 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:39:20.266 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:39:20.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:20.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:21.180 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:21.180 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:21.670 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:22.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:22.356 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:23.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:23.613 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014714, gin 1, member 0 [ CSSD]2013-05-23 16:39:23.613 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:39:23.613 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:39:23.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:23.619 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:39:23.619 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:39:23.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:23.815 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:39:23.815 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1710 (0x7f431bff4710) [ CSSD]2013-05-23 16:39:23.815 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4710 (RPC#1710) tag(6ae002c) sent to node 1 [ CSSD]2013-05-23 16:39:23.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:23.816 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1710 (0x7f431bff4710) [ CSSD]2013-05-23 16:39:23.816 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4710 (RPC#1710) state 6, flags 0x202 [ CSSD]2013-05-23 16:39:23.816 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4710, ret 0, client (nil) [ CSSD]2013-05-23 16:39:23.816 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1710 [ CSSD]2013-05-23 16:39:24.053 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:39:24.054 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:39:24.054 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:39:24.054 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1246d0) pid(23769) version 11:1:1:4 [ CSSD]2013-05-23 16:39:24.060 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1246d0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:39:24.060 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:39:24.061 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c109f70, requested num -1 [ CSSD]2013-05-23 16:39:24.061 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:24.061 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1711 (0x7f431bff4790) [ CSSD]2013-05-23 16:39:24.061 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4790 (RPC#1711) tag(6af002c) sent to node 1 [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06af002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014715, gin 1, member 0 [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4790 (RPC#1711) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4790, ret 0, client 0x7f431c0cf720 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:39:24.061 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1711 [ CSSD]2013-05-23 16:39:24.068 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:39:24.068 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:24.068 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1215d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:24.068 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1712 (0x7f431bff4810) [ CSSD]2013-05-23 16:39:24.068 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4810 (RPC#1712) tag(6b0002c) sent to node 1 [ CSSD]2013-05-23 16:39:24.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06b0002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:24.068 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:24.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4810 (RPC#1712) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:24.068 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4810, ret 0, client 0x7f431c0cf720 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:24.069 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1712 [ CSSD]2013-05-23 16:39:24.069 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1215d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:39:24.271 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:39:24.271 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c108e60) skgpid ospid 23769 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:39:24.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:24.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:25.730 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:26.183 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:26.183 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:26.326 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:39:26.326 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:39:26.328 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:39:26.329 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:39:26.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:26.409 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014716, gin 1, member 0 [ CSSD]2013-05-23 16:39:26.409 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f3340/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:39:26.409 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:39:26.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:26.417 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:39:26.417 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:39:26.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:26.455 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:27.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:28.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:28.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:28.613 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:39:28.614 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:39:28.614 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:39:28.614 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1246d0) pid(23803) version 11:1:1:4 [ CSSD]2013-05-23 16:39:28.619 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1246d0), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 16:39:28.619 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:39:28.619 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5900, requested num -1 [ CSSD]2013-05-23 16:39:28.619 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:28.619 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1713 (0x7f431bff4890) [ CSSD]2013-05-23 16:39:28.620 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4890 (RPC#1713) tag(6b1002c) sent to node 1 [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06b1002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014717, gin 1, member 0 [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4890 (RPC#1713) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4890, ret 0, client 0x7f431c12e6d0 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 16:39:28.620 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1713 [ CSSD]2013-05-23 16:39:28.626 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:39:28.626 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:28.626 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1215d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:28.626 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1714 (0x7f431bff4910) [ CSSD]2013-05-23 16:39:28.626 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4910 (RPC#1714) tag(6b2002c) sent to node 1 [ CSSD]2013-05-23 16:39:28.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06b2002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:28.627 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:28.627 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4910 (RPC#1714) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:28.627 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4910, ret 0, client 0x7f431c12e6d0 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:28.627 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1714 [ CSSD]2013-05-23 16:39:28.627 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1215d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:39:28.829 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:39:28.829 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c0e6720) skgpid ospid 23803 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:39:29.748 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:30.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:30.473 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:31.186 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:31.186 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:31.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:32.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:32.477 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:33.765 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:33.846 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:39:33.846 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1715 (0x7f431bff4990) [ CSSD]2013-05-23 16:39:33.847 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4990 (RPC#1715) tag(6b3002c) sent to node 1 [ CSSD]2013-05-23 16:39:33.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:33.847 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1715 (0x7f431bff4990) [ CSSD]2013-05-23 16:39:33.847 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4990 (RPC#1715) state 6, flags 0x202 [ CSSD]2013-05-23 16:39:33.847 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4990, ret 0, client (nil) [ CSSD]2013-05-23 16:39:33.847 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1715 [ CSSD]2013-05-23 16:39:34.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:34.490 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:35.771 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:36.189 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:36.189 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:36.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:36.495 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:37.822 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:38.500 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:38.500 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:38.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:38.859 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014718, gin 1, member 0 [ CSSD]2013-05-23 16:39:38.859 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:39:38.859 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:39:38.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:38.865 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:39:38.865 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:39:39.827 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:40.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:40.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:41.192 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:41.192 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:41.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:42.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:42.578 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:43.846 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:43.864 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:39:43.865 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:39:43.866 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:39:43.866 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c1246d0) pid(23943) version 11:1:1:4 [ CSSD]2013-05-23 16:39:43.875 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:39:43.876 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1716 (0x7f431bff4a10) [ CSSD]2013-05-23 16:39:43.876 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4a10 (RPC#1716) tag(6b4002c) sent to node 1 [ CSSD]2013-05-23 16:39:43.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:43.876 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1716 (0x7f431bff4a10) [ CSSD]2013-05-23 16:39:43.876 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4a10 (RPC#1716) state 6, flags 0x202 [ CSSD]2013-05-23 16:39:43.876 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4a10, ret 0, client (nil) [ CSSD]2013-05-23 16:39:43.877 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1716 [ CSSD]2013-05-23 16:39:43.882 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1246d0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:39:43.882 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:39:43.882 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:39:43.882 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:43.882 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1717 (0x7f431bff4a90) [ CSSD]2013-05-23 16:39:43.882 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4a90 (RPC#1717) tag(6b5002c) sent to node 1 [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06b5002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014719, gin 1, member 0 [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4a90 (RPC#1717) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4a90, ret 0, client 0x7f431c0e6e30 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:39:43.883 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1717 [ CSSD]2013-05-23 16:39:43.890 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:39:43.890 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:43.890 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1215d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:43.890 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1718 (0x7f431bff4b10) [ CSSD]2013-05-23 16:39:43.890 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4b10 (RPC#1718) tag(6b6002c) sent to node 1 [ CSSD]2013-05-23 16:39:43.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06b6002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:43.890 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:43.890 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4b10 (RPC#1718) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:43.890 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4b10, ret 0, client 0x7f431c0e6e30 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:43.890 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1718 [ CSSD]2013-05-23 16:39:43.890 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1215d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:39:44.093 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:39:44.093 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c0e5900) skgpid ospid 23943 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:39:44.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:44.651 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:45.871 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:46.196 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:46.196 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:46.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:46.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:47.923 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:48.681 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:48.681 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:49.929 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:50.199 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:50.199 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:39:50.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:50.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:51.934 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:52.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:52.779 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:53.906 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:39:53.906 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1719 (0x7f431bff4b90) [ CSSD]2013-05-23 16:39:53.906 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4b90 (RPC#1719) tag(6b7002c) sent to node 1 [ CSSD]2013-05-23 16:39:53.907 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:53.907 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1719 (0x7f431bff4b90) [ CSSD]2013-05-23 16:39:53.907 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4b90 (RPC#1719) state 6, flags 0x202 [ CSSD]2013-05-23 16:39:53.907 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4b90, ret 0, client (nil) [ CSSD]2013-05-23 16:39:53.907 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1719 [ CSSD]2013-05-23 16:39:53.945 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:54.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:54.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014720, gin 1, member 0 [ CSSD]2013-05-23 16:39:54.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:39:54.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:39:54.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:54.142 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:39:54.142 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:39:54.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:54.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:55.202 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:39:55.202 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:39:55.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:56.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:56.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:58.023 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:39:58.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:39:58.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:39:59.130 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:39:59.131 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:39:59.132 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:39:59.132 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c1246d0) pid(24087) version 11:1:1:4 [ CSSD]2013-05-23 16:39:59.138 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1246d0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:39:59.138 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:39:59.138 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:39:59.138 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:59.138 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1720 (0x7f431bff4c10) [ CSSD]2013-05-23 16:39:59.138 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4c10 (RPC#1720) tag(6b8002c) sent to node 1 [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06b8002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014721, gin 1, member 0 [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4c10 (RPC#1720) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4c10, ret 0, client 0x7f431c0cf720 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:59.138 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:39:59.139 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1720 [ CSSD]2013-05-23 16:39:59.145 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:39:59.145 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:59.145 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1215d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:39:59.145 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1721 (0x7f431bff4c90) [ CSSD]2013-05-23 16:39:59.145 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4c90 (RPC#1721) tag(6b9002c) sent to node 1 [ CSSD]2013-05-23 16:39:59.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06b9002c) incarnation(174631051) [ CSSD]2013-05-23 16:39:59.146 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:39:59.146 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4c90 (RPC#1721) state 6, flags 0x100 [ CSSD]2013-05-23 16:39:59.146 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4c90, ret 0, client 0x7f431c0cf720 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:39:59.146 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1721 [ CSSD]2013-05-23 16:39:59.146 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1215d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:39:59.349 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:39:59.349 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c0e5200) skgpid ospid 24087 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:00.048 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:00.205 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:00.205 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:00.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:00.864 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:02.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:02.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:02.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:03.936 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:40:03.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1722 (0x7f431bff4d10) [ CSSD]2013-05-23 16:40:03.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4d10 (RPC#1722) tag(6ba002c) sent to node 1 [ CSSD]2013-05-23 16:40:03.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:03.937 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1722 (0x7f431bff4d10) [ CSSD]2013-05-23 16:40:03.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4d10 (RPC#1722) state 6, flags 0x202 [ CSSD]2013-05-23 16:40:03.937 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4d10, ret 0, client (nil) [ CSSD]2013-05-23 16:40:03.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1722 [ CSSD]2013-05-23 16:40:04.065 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:04.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:04.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:05.208 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:05.208 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:06.071 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:06.907 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:06.907 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:07.994 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:40:07.995 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:40:07.995 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:40:07.995 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c1246d0) pid(24395) version 11:1:1:4 [ CSSD]2013-05-23 16:40:08.123 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:08.550 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:40:08.550 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c109f70) skgpid ospid 24395 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:08.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:08.913 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:09.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:09.347 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014722, gin 1, member 0 [ CSSD]2013-05-23 16:40:09.347 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3340/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:40:09.347 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:40:09.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:09.353 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:40:09.353 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1bb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:40:10.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:10.211 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:10.211 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:10.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:10.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:12.152 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:12.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:12.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:13.956 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:40:13.956 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1723 (0x7f431bff4d90) [ CSSD]2013-05-23 16:40:13.956 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4d90 (RPC#1723) tag(6bb002c) sent to node 1 [ CSSD]2013-05-23 16:40:13.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:13.957 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1723 (0x7f431bff4d90) [ CSSD]2013-05-23 16:40:13.957 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4d90 (RPC#1723) state 6, flags 0x202 [ CSSD]2013-05-23 16:40:13.957 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4d90, ret 0, client (nil) [ CSSD]2013-05-23 16:40:13.957 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1723 [ CSSD]2013-05-23 16:40:14.164 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:14.213 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:14.214 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:40:14.381 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:40:14.382 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:40:14.383 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:40:14.383 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c1246d0) pid(24560) version 11:1:1:4 [ CSSD]2013-05-23 16:40:14.389 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c1246d0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:40:14.389 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:40:14.389 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:40:14.389 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:14.389 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1724 (0x7f431bff4e10) [ CSSD]2013-05-23 16:40:14.389 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4e10 (RPC#1724) tag(6bc002c) sent to node 1 [ CSSD]2013-05-23 16:40:14.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06bc002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:14.389 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014723, gin 1, member 0 [ CSSD]2013-05-23 16:40:14.389 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:40:14.389 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:40:14.389 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:40:14.390 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4e10 (RPC#1724) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:14.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4e10, ret 0, client 0x7f431c0cf720 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:40:14.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 16:40:14.390 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1724 [ CSSD]2013-05-23 16:40:14.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:40:14.396 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:14.396 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1215d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:14.396 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1725 (0x7f431bff4e90) [ CSSD]2013-05-23 16:40:14.396 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4e90 (RPC#1725) tag(6bd002c) sent to node 1 [ CSSD]2013-05-23 16:40:14.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06bd002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:14.396 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:14.396 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4e90 (RPC#1725) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:14.396 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff4e90, ret 0, client 0x7f431c0cf720 member 0x7f431c1215d0 [ CSSD]2013-05-23 16:40:14.396 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1725 [ CSSD]2013-05-23 16:40:14.396 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1215d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:40:14.599 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:40:14.599 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c0840c0) skgpid ospid 24560 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:15.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:15.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:16.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:17.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:17.106 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:18.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:19.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:19.161 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:19.217 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:19.217 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:19.436 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ed590) [ CSSD]2013-05-23 16:40:19.436 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ed590), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:40:19.436 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ed590), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:40:19.436 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:40:19.437 [1191196992] >TRACE: clssgmDeadProc: proc 0x23191c0 [ CSSD]2013-05-23 16:40:19.437 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23191c0) con(0x22e9fa0) skgpid 19309 ospid 19309 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:19.448 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2328d60) [ CSSD]2013-05-23 16:40:19.448 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2328d60), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:40:19.448 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2328d60), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:40:19.448 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:40:19.448 [1191196992] >TRACE: clssgmDeadProc: proc 0x23286e0 [ CSSD]2013-05-23 16:40:19.448 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23286e0) con(0x2326460) skgpid 19311 ospid 19311 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:19.449 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:40:19.449 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23029c0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:40:19.449 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23029c0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:40:19.449 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:40:19.449 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317a30 [ CSSD]2013-05-23 16:40:19.450 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317a30) con(0x22f4ac0) skgpid 19313 ospid 19313 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:20.230 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:20.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:20.515 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014724, gin 1, member 0 [ CSSD]2013-05-23 16:40:20.515 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:40:20.515 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:40:20.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:20.522 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:40:20.522 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:40:21.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:21.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:22.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:22.272 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1246d0) allocated [ CSSD]2013-05-23 16:40:22.272 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:40:22.273 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1246d0 - 1,2,3 [ CSSD]2013-05-23 16:40:22.273 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c1246d0) pid(24641/24641) version 11:1:1:4 [ CSSD]2013-05-23 16:40:22.279 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c1246d0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 16:40:22.279 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:40:22.279 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:40:22.279 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:40:23.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:23.171 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:23.976 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:40:23.976 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1726 (0x7f431bff4f10) [ CSSD]2013-05-23 16:40:23.977 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4f10 (RPC#1726) tag(6be002c) sent to node 1 [ CSSD]2013-05-23 16:40:23.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:23.977 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1726 (0x7f431bff4f10) [ CSSD]2013-05-23 16:40:23.977 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4f10 (RPC#1726) state 6, flags 0x202 [ CSSD]2013-05-23 16:40:23.977 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff4f10, ret 0, client (nil) [ CSSD]2013-05-23 16:40:23.977 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1726 [ CSSD]2013-05-23 16:40:24.220 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:24.220 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:24.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:24.311 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:40:24.311 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:40:24.312 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:40:24.312 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c1348f0) pid(24671) version 11:1:1:4 [ CSSD]2013-05-23 16:40:24.319 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:24.319 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:24.319 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:40:24.319 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:24.319 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1727 (0x7f431bff4f90) [ CSSD]2013-05-23 16:40:24.319 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff4f90 (RPC#1727) tag(6bf002c) sent to node 1 [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06bf002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014725, gin 1, member 0 [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff4f90 (RPC#1727) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff4f90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129940 [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 16:40:24.320 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1727 [ CSSD]2013-05-23 16:40:24.326 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:24.326 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:24.326 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129940) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:24.326 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1728 (0x7f431bff5010) [ CSSD]2013-05-23 16:40:24.326 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5010 (RPC#1728) tag(6c0002c) sent to node 1 [ CSSD]2013-05-23 16:40:24.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06c0002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:24.327 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:24.327 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5010 (RPC#1728) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:24.327 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5010, ret 0, client 0x7f431c12e6d0 member 0x7f431c129940 [ CSSD]2013-05-23 16:40:24.327 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1728 [ CSSD]2013-05-23 16:40:24.327 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129940) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:40:24.529 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:40:24.529 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0840c0) skgpid ospid 24671 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:24.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:24.595 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014726, gin 1, member 0 [ CSSD]2013-05-23 16:40:24.595 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:40:24.595 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:40:24.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:24.601 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:40:24.601 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:40:24.855 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c1246d0), client(2/0x7f431c0e6e30) [ CSSD]2013-05-23 16:40:24.855 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:40:24.855 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:40:24.855 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:40:25.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:25.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:26.273 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:27.188 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:27.188 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:28.275 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:29.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:29.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:29.223 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:29.223 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:29.632 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136070) allocated [ CSSD]2013-05-23 16:40:29.633 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:40:29.634 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136070 - 1,2,3 [ CSSD]2013-05-23 16:40:29.634 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c136070) pid(24717) version 11:1:1:4 [ CSSD]2013-05-23 16:40:29.639 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c136070), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:29.639 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:29.639 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:40:29.639 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:29.639 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1729 (0x7f431bff5090) [ CSSD]2013-05-23 16:40:29.639 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5090 (RPC#1729) tag(6c1002c) sent to node 1 [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06c1002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014727, gin 1, member 0 [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129840/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5090 (RPC#1729) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5090, ret 0, client 0x7f431c12e6d0 member 0x7f431c120ad0 [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 16:40:29.640 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1729 [ CSSD]2013-05-23 16:40:29.647 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:29.647 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:29.647 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c120ad0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:29.647 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1730 (0x7f431bff5110) [ CSSD]2013-05-23 16:40:29.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5110 (RPC#1730) tag(6c2002c) sent to node 1 [ CSSD]2013-05-23 16:40:29.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06c2002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:29.647 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:29.647 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5110 (RPC#1730) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:29.647 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5110, ret 0, client 0x7f431c12e6d0 member 0x7f431c120ad0 [ CSSD]2013-05-23 16:40:29.647 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1730 [ CSSD]2013-05-23 16:40:29.647 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c120ad0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:40:29.850 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136070 [ CSSD]2013-05-23 16:40:29.850 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136070) con(0x7f431c108e60) skgpid ospid 24717 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:30.300 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:31.206 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:31.206 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:32.305 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:33.157 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136070) allocated [ CSSD]2013-05-23 16:40:33.158 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:40:33.159 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136070 - 1,2,3 [ CSSD]2013-05-23 16:40:33.159 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c136070) pid(24643/24643) version 11:1:1:4 [ CSSD]2013-05-23 16:40:33.164 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c136070), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:33.164 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:40:33.164 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:40:33.164 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:40:33.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:33.211 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:33.226 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:33.226 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:40:33.997 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:40:33.997 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1731 (0x7f431bff5190) [ CSSD]2013-05-23 16:40:33.997 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5190 (RPC#1731) tag(6c3002c) sent to node 1 [ CSSD]2013-05-23 16:40:33.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:33.998 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1731 (0x7f431bff5190) [ CSSD]2013-05-23 16:40:33.998 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5190 (RPC#1731) state 6, flags 0x202 [ CSSD]2013-05-23 16:40:33.998 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5190, ret 0, client (nil) [ CSSD]2013-05-23 16:40:33.998 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1731 [ CSSD]2013-05-23 16:40:34.338 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:35.263 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:35.263 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:36.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:37.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:37.268 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:38.229 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:38.229 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:38.355 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:39.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:39.281 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:39.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:39.850 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014728, gin 1, member 0 [ CSSD]2013-05-23 16:40:39.850 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:40:39.850 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:40:39.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:39.856 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:40:39.856 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:40:40.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:41.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:41.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:42.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:43.231 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:43.232 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:43.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:43.291 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:44.037 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:40:44.037 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1732 (0x7f431bff5210) [ CSSD]2013-05-23 16:40:44.037 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5210 (RPC#1732) tag(6c4002c) sent to node 1 [ CSSD]2013-05-23 16:40:44.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:44.038 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1732 (0x7f431bff5210) [ CSSD]2013-05-23 16:40:44.038 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5210 (RPC#1732) state 6, flags 0x202 [ CSSD]2013-05-23 16:40:44.038 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5210, ret 0, client (nil) [ CSSD]2013-05-23 16:40:44.038 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1732 [ CSSD]2013-05-23 16:40:44.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:44.882 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:40:44.883 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:40:44.884 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:40:44.884 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(24864) version 11:1:1:4 [ CSSD]2013-05-23 16:40:44.890 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:40:44.890 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:40:44.890 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c1214c0, requested num -1 [ CSSD]2013-05-23 16:40:44.890 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:44.890 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1733 (0x7f431bff5290) [ CSSD]2013-05-23 16:40:44.890 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5290 (RPC#1733) tag(6c5002c) sent to node 1 [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06c5002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014729, gin 1, member 0 [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5290 (RPC#1733) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5290, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:40:44.891 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1733 [ CSSD]2013-05-23 16:40:44.899 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:40:44.899 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:44.899 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:44.899 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1734 (0x7f431bff5310) [ CSSD]2013-05-23 16:40:44.899 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5310 (RPC#1734) tag(6c6002c) sent to node 1 [ CSSD]2013-05-23 16:40:44.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06c6002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:44.899 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:44.899 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5310 (RPC#1734) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:44.899 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5310, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:40:44.900 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1734 [ CSSD]2013-05-23 16:40:44.900 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:40:45.102 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:40:45.102 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 24864 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:45.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:45.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:46.383 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:47.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:47.389 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:48.235 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:48.235 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:48.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:49.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:49.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:50.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:51.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:51.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:52.238 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:52.238 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:40:52.448 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:53.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:53.490 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:54.067 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:40:54.067 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1735 (0x7f431bff5390) [ CSSD]2013-05-23 16:40:54.067 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5390 (RPC#1735) tag(6c7002c) sent to node 1 [ CSSD]2013-05-23 16:40:54.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:54.068 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1735 (0x7f431bff5390) [ CSSD]2013-05-23 16:40:54.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5390 (RPC#1735) state 6, flags 0x202 [ CSSD]2013-05-23 16:40:54.068 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5390, ret 0, client (nil) [ CSSD]2013-05-23 16:40:54.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1735 [ CSSD]2013-05-23 16:40:54.459 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:55.099 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:55.099 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014730, gin 1, member 0 [ CSSD]2013-05-23 16:40:55.099 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:40:55.099 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:40:55.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:55.104 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:40:55.104 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:40:55.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:55.584 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:56.484 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:57.241 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:40:57.241 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:40:57.323 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:40:57.325 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:40:57.325 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:40:57.325 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(25008) version 11:1:1:4 [ CSSD]2013-05-23 16:40:57.332 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:40:57.332 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:40:57.332 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c1214c0, requested num -1 [ CSSD]2013-05-23 16:40:57.332 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:57.332 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1736 (0x7f431bff5410) [ CSSD]2013-05-23 16:40:57.332 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5410 (RPC#1736) tag(6c8002c) sent to node 1 [ CSSD]2013-05-23 16:40:57.332 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06c8002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:57.332 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014731, gin 1, member 0 [ CSSD]2013-05-23 16:40:57.332 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:40:57.333 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:40:57.333 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:40:57.333 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5410 (RPC#1736) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:57.333 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5410, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:40:57.333 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:40:57.333 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1736 [ CSSD]2013-05-23 16:40:57.339 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:40:57.339 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:57.339 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:40:57.339 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1737 (0x7f431bff5490) [ CSSD]2013-05-23 16:40:57.339 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5490 (RPC#1737) tag(6c9002c) sent to node 1 [ CSSD]2013-05-23 16:40:57.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06c9002c) incarnation(174631051) [ CSSD]2013-05-23 16:40:57.340 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:40:57.340 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5490 (RPC#1737) state 6, flags 0x100 [ CSSD]2013-05-23 16:40:57.340 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5490, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:40:57.340 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1737 [ CSSD]2013-05-23 16:40:57.340 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:40:57.542 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:40:57.542 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 25008 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:40:57.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:57.589 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:40:58.536 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:40:59.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:40:59.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:00.134 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:41:00.135 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:41:00.135 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:41:00.135 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(25036) version 11:1:1:4 [ CSSD]2013-05-23 16:41:00.142 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:41:00.142 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:00.142 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:41:00.142 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:00.142 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1738 (0x7f431bff5510) [ CSSD]2013-05-23 16:41:00.142 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5510 (RPC#1738) tag(6ca002c) sent to node 1 [ CSSD]2013-05-23 16:41:00.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06ca002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:00.142 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014732, gin 1, member 0 [ CSSD]2013-05-23 16:41:00.142 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:41:00.143 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:41:00.143 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:41:00.143 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5510 (RPC#1738) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:00.143 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5510, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:41:00.143 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:41:00.143 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1738 [ CSSD]2013-05-23 16:41:00.149 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:00.149 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:00.149 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:00.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1739 (0x7f431bff5590) [ CSSD]2013-05-23 16:41:00.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5590 (RPC#1739) tag(6cb002c) sent to node 1 [ CSSD]2013-05-23 16:41:00.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06cb002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:00.149 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:00.149 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5590 (RPC#1739) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:00.149 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5590, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:41:00.149 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1739 [ CSSD]2013-05-23 16:41:00.149 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:41:00.352 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:41:00.352 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 25036 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:00.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:01.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:01.607 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:02.244 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:02.244 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:02.568 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:03.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:03.612 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:04.097 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:41:04.097 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1740 (0x7f431bff5610) [ CSSD]2013-05-23 16:41:04.097 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5610 (RPC#1740) tag(6cc002c) sent to node 1 [ CSSD]2013-05-23 16:41:04.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:04.098 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1740 (0x7f431bff5610) [ CSSD]2013-05-23 16:41:04.098 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5610 (RPC#1740) state 6, flags 0x202 [ CSSD]2013-05-23 16:41:04.098 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5610, ret 0, client (nil) [ CSSD]2013-05-23 16:41:04.098 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1740 [ CSSD]2013-05-23 16:41:04.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:05.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:05.664 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:06.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:07.247 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:07.248 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:07.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:07.668 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:08.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:09.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:09.672 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:10.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:10.353 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014733, gin 1, member 0 [ CSSD]2013-05-23 16:41:10.353 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:41:10.353 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:41:10.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:10.359 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:41:10.359 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:41:10.661 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:11.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:11.677 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:12.250 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:12.250 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:12.674 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:13.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:13.682 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:14.120 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:41:14.120 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1741 (0x7f431bff5690) [ CSSD]2013-05-23 16:41:14.120 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5690 (RPC#1741) tag(6cd002c) sent to node 1 [ CSSD]2013-05-23 16:41:14.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:14.121 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1741 (0x7f431bff5690) [ CSSD]2013-05-23 16:41:14.121 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5690 (RPC#1741) state 6, flags 0x202 [ CSSD]2013-05-23 16:41:14.121 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5690, ret 0, client (nil) [ CSSD]2013-05-23 16:41:14.121 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1741 [ CSSD]2013-05-23 16:41:14.700 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:15.384 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:41:15.385 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:41:15.385 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:41:15.385 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(25199) version 11:1:1:4 [ CSSD]2013-05-23 16:41:15.392 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:41:15.392 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:41:15.392 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c1214c0, requested num -1 [ CSSD]2013-05-23 16:41:15.392 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:15.392 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1742 (0x7f431bff5710) [ CSSD]2013-05-23 16:41:15.392 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5710 (RPC#1742) tag(6ce002c) sent to node 1 [ CSSD]2013-05-23 16:41:15.392 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06ce002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:15.392 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014734, gin 1, member 0 [ CSSD]2013-05-23 16:41:15.392 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:41:15.392 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:41:15.393 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:41:15.393 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5710 (RPC#1742) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:15.393 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5710, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:41:15.393 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:41:15.393 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1742 [ CSSD]2013-05-23 16:41:15.399 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:41:15.399 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:15.399 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:15.399 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1743 (0x7f431bff5790) [ CSSD]2013-05-23 16:41:15.399 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5790 (RPC#1743) tag(6cf002c) sent to node 1 [ CSSD]2013-05-23 16:41:15.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06cf002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:15.400 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:15.400 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5790 (RPC#1743) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:15.400 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5790, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:41:15.400 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1743 [ CSSD]2013-05-23 16:41:15.400 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:41:15.602 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:41:15.602 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 25199 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:15.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:15.695 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:16.253 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:16.253 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:41:16.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:17.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:17.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:18.513 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:41:18.514 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:41:18.515 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:41:18.515 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(25262) version 11:1:1:4 [ CSSD]2013-05-23 16:41:18.532 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:41:18.532 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:18.532 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:41:18.532 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:41:18.532 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1744 (0x7f431bff5810) [ CSSD]2013-05-23 16:41:18.532 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5810 (RPC#1744) tag(6d0002c) sent to node 1 [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06d0002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014735, gin 1, member 0 [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5810 (RPC#1744) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5810, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:41:18.533 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1744 [ CSSD]2013-05-23 16:41:18.539 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:18.539 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:41:18.539 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:41:18.539 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1745 (0x7f431bff5890) [ CSSD]2013-05-23 16:41:18.540 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5890 (RPC#1745) tag(6d1002c) sent to node 1 [ CSSD]2013-05-23 16:41:18.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06d1002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:18.540 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:41:18.540 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5890 (RPC#1745) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:18.540 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5890, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:41:18.540 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1745 [ CSSD]2013-05-23 16:41:18.540 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:41:18.651 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092800) allocated [ CSSD]2013-05-23 16:41:18.652 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:41:18.652 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092800 - 1,2,3 [ CSSD]2013-05-23 16:41:18.652 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c092800) pid(25267) version 11:1:1:4 [ CSSD]2013-05-23 16:41:18.665 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:41:18.667 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:41:18.670 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092800 [ CSSD]2013-05-23 16:41:18.670 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092800) con(0x7f431c0840c0) skgpid ospid 25267 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:18.737 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:18.743 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:41:18.743 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 25262 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:19.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:19.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:20.761 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:20.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:20.762 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014736, gin 1, member 0 [ CSSD]2013-05-23 16:41:20.762 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:41:20.762 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:41:20.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:20.767 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:41:20.767 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:41:21.255 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:21.255 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:21.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:21.817 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:22.774 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:23.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:23.821 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:24.017 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2303710) [ CSSD]2013-05-23 16:41:24.017 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2303710), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:41:24.017 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2303710), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:41:24.017 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 16:41:24.026 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1730) [ CSSD]2013-05-23 16:41:24.026 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1730), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:41:24.026 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1730), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:41:24.026 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:41:24.027 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:41:24.027 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2312880) skgpid 12802 ospid 12802 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:24.147 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:41:24.147 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1746 (0x7f431bff5910) [ CSSD]2013-05-23 16:41:24.147 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5910 (RPC#1746) tag(6d2002c) sent to node 1 [ CSSD]2013-05-23 16:41:24.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:24.148 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1746 (0x7f431bff5910) [ CSSD]2013-05-23 16:41:24.148 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5910 (RPC#1746) state 6, flags 0x202 [ CSSD]2013-05-23 16:41:24.148 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5910, ret 0, client (nil) [ CSSD]2013-05-23 16:41:24.148 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1746 [ CSSD]2013-05-23 16:41:24.574 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:41:24.575 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:41:24.576 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:41:24.576 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(25338) version 11:1:1:4 [ CSSD]2013-05-23 16:41:24.593 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:41:24.593 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:24.593 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:41:24.593 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:24.593 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1747 (0x7f431bff5990) [ CSSD]2013-05-23 16:41:24.593 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5990 (RPC#1747) tag(6d3002c) sent to node 1 [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06d3002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014737, gin 1, member 0 [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5990 (RPC#1747) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5990, ret 0, client 0x7f431c091070 member 0x7f431c134520 [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:41:24.594 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1747 [ CSSD]2013-05-23 16:41:24.601 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:24.601 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:24.601 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:24.601 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1748 (0x7f431bff5a10) [ CSSD]2013-05-23 16:41:24.601 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5a10 (RPC#1748) tag(6d4002c) sent to node 1 [ CSSD]2013-05-23 16:41:24.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06d4002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:24.601 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:24.601 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5a10 (RPC#1748) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:24.601 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5a10, ret 0, client 0x7f431c091070 member 0x7f431c134520 [ CSSD]2013-05-23 16:41:24.601 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1748 [ CSSD]2013-05-23 16:41:24.601 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:41:24.780 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:24.804 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:41:24.804 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 25338 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:25.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:25.608 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014738, gin 1, member 0 [ CSSD]2013-05-23 16:41:25.608 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:41:25.608 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:41:25.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:25.615 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:41:25.615 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231edc0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:41:25.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:25.876 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:26.258 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:26.258 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:26.785 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:27.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:27.881 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:28.837 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:29.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:29.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:30.635 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:41:30.636 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:41:30.636 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:41:30.636 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(25397) version 11:1:1:4 [ CSSD]2013-05-23 16:41:30.642 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:41:30.642 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:30.642 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:41:30.642 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:30.642 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1749 (0x7f431bff5a90) [ CSSD]2013-05-23 16:41:30.642 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5a90 (RPC#1749) tag(6d5002c) sent to node 1 [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06d5002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014739, gin 1, member 0 [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5a90 (RPC#1749) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5a90, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:41:30.643 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1749 [ CSSD]2013-05-23 16:41:30.650 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:41:30.650 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:30.650 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:30.650 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1750 (0x7f431bff5b10) [ CSSD]2013-05-23 16:41:30.650 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5b10 (RPC#1750) tag(6d6002c) sent to node 1 [ CSSD]2013-05-23 16:41:30.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06d6002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:30.650 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:30.650 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5b10 (RPC#1750) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:30.650 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5b10, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:41:30.650 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1750 [ CSSD]2013-05-23 16:41:30.650 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:41:30.844 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:30.853 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:41:30.853 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 25397 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:31.261 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:31.261 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:31.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:31.898 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:32.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:33.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:33.903 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:34.174 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:41:34.175 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1751 (0x7f431bff5b90) [ CSSD]2013-05-23 16:41:34.175 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5b90 (RPC#1751) tag(6d7002c) sent to node 1 [ CSSD]2013-05-23 16:41:34.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:34.176 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1751 (0x7f431bff5b90) [ CSSD]2013-05-23 16:41:34.176 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5b90 (RPC#1751) state 6, flags 0x202 [ CSSD]2013-05-23 16:41:34.176 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5b90, ret 0, client (nil) [ CSSD]2013-05-23 16:41:34.176 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1751 [ CSSD]2013-05-23 16:41:34.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:35.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:35.916 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:36.264 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:36.264 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:36.885 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:37.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:37.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:38.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:39.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:39.974 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:40.267 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:40.267 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:41:40.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:40.856 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014740, gin 1, member 0 [ CSSD]2013-05-23 16:41:40.856 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:41:40.856 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:41:40.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:40.863 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:41:40.863 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:41:40.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:41.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:41.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:42.946 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:43.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:43.984 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:44.200 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:41:44.200 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1752 (0x7f431bff5c10) [ CSSD]2013-05-23 16:41:44.200 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5c10 (RPC#1752) tag(6d8002c) sent to node 1 [ CSSD]2013-05-23 16:41:44.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:44.201 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1752 (0x7f431bff5c10) [ CSSD]2013-05-23 16:41:44.201 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5c10 (RPC#1752) state 6, flags 0x202 [ CSSD]2013-05-23 16:41:44.201 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5c10, ret 0, client (nil) [ CSSD]2013-05-23 16:41:44.201 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1752 [ CSSD]2013-05-23 16:41:44.952 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:45.270 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:45.270 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:45.894 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:41:45.895 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:41:45.896 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:41:45.896 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(25559) version 11:1:1:4 [ CSSD]2013-05-23 16:41:45.902 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:41:45.902 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:41:45.902 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c129b20, requested num -1 [ CSSD]2013-05-23 16:41:45.902 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:45.902 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1753 (0x7f431bff5c90) [ CSSD]2013-05-23 16:41:45.902 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5c90 (RPC#1753) tag(6d9002c) sent to node 1 [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06d9002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014741, gin 1, member 0 [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5c90 (RPC#1753) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5c90, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:41:45.903 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1753 [ CSSD]2013-05-23 16:41:45.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:41:45.910 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:45.910 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:41:45.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1754 (0x7f431bff5d10) [ CSSD]2013-05-23 16:41:45.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5d10 (RPC#1754) tag(6da002c) sent to node 1 [ CSSD]2013-05-23 16:41:45.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06da002c) incarnation(174631051) [ CSSD]2013-05-23 16:41:45.910 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:41:45.910 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5d10 (RPC#1754) state 6, flags 0x100 [ CSSD]2013-05-23 16:41:45.910 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5d10, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:41:45.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1754 [ CSSD]2013-05-23 16:41:45.911 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:41:45.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:45.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:46.113 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:41:46.114 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 25559 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:41:46.958 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:48.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:48.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:48.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:50.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:50.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:50.273 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:50.273 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:50.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:51.073 [1191196992] >TRACE: clssgmAllocProc: (0x2303710) allocated [ CSSD]2013-05-23 16:41:51.074 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:41:51.074 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2303710 - 1,2,3 [ CSSD]2013-05-23 16:41:51.074 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2a10) proc(0x2303710) pid(25335/25335) version 11:1:1:4 [ CSSD]2013-05-23 16:41:51.081 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2303710), client(1/0x2312950) [ CSSD]2013-05-23 16:41:51.081 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:41:51.081 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:41:51.081 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:41:52.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:52.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:53.046 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:54.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:54.072 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:54.230 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:41:54.231 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1755 (0x7f431bff5d90) [ CSSD]2013-05-23 16:41:54.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5d90 (RPC#1755) tag(6db002c) sent to node 1 [ CSSD]2013-05-23 16:41:54.231 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:54.231 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1755 (0x7f431bff5d90) [ CSSD]2013-05-23 16:41:54.231 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5d90 (RPC#1755) state 6, flags 0x202 [ CSSD]2013-05-23 16:41:54.231 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5d90, ret 0, client (nil) [ CSSD]2013-05-23 16:41:54.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1755 [ CSSD]2013-05-23 16:41:55.052 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:55.276 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:55.277 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:41:56.077 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:56.077 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:56.103 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:56.103 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014742, gin 1, member 0 [ CSSD]2013-05-23 16:41:56.103 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:41:56.103 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:41:56.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:56.110 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:41:56.110 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:41:57.077 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:58.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:41:58.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:41:59.089 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:41:59.279 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:41:59.279 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:42:00.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:00.194 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:01.094 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:01.155 [1191196992] >TRACE: clssgmAllocProc: (0x2302bf0) allocated [ CSSD]2013-05-23 16:42:01.156 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:42:01.156 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302bf0 - 1,2,3 [ CSSD]2013-05-23 16:42:01.156 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7cd0) proc(0x2302bf0) pid(25704) version 11:1:1:4 [ CSSD]2013-05-23 16:42:01.163 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2302bf0), client(1/0x23029c0) [ CSSD]2013-05-23 16:42:01.163 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:42:01.163 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23029c0 with con 0x2317960, requested num -1 [ CSSD]2013-05-23 16:42:01.163 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:01.163 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1756 (0x7f431bff5e10) [ CSSD]2013-05-23 16:42:01.163 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5e10 (RPC#1756) tag(6dc002c) sent to node 1 [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06dc002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014743, gin 1, member 0 [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5e10 (RPC#1756) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5e10, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:01.163 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23029c0 [ CSSD]2013-05-23 16:42:01.164 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1756 [ CSSD]2013-05-23 16:42:01.170 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:42:01.170 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23029c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:01.170 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:01.170 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1757 (0x7f431bff5e90) [ CSSD]2013-05-23 16:42:01.170 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5e90 (RPC#1757) tag(6dd002c) sent to node 1 [ CSSD]2013-05-23 16:42:01.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06dd002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:01.171 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:01.171 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5e90 (RPC#1757) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:01.171 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff5e90, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:01.171 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1757 [ CSSD]2013-05-23 16:42:01.171 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:01.373 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302bf0 [ CSSD]2013-05-23 16:42:01.373 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302bf0) con(0x22d7cd0) skgpid ospid 25704 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:02.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:02.199 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:03.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:04.261 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:42:04.261 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1758 (0x7f431bff5f10) [ CSSD]2013-05-23 16:42:04.262 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5f10 (RPC#1758) tag(6de002c) sent to node 1 [ CSSD]2013-05-23 16:42:04.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:04.262 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1758 (0x7f431bff5f10) [ CSSD]2013-05-23 16:42:04.262 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5f10 (RPC#1758) state 6, flags 0x202 [ CSSD]2013-05-23 16:42:04.262 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff5f10, ret 0, client (nil) [ CSSD]2013-05-23 16:42:04.262 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1758 [ CSSD]2013-05-23 16:42:04.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:04.273 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:04.281 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:04.282 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:05.152 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:06.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:06.279 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:07.176 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:07.385 [1191196992] >TRACE: clssgmAllocProc: (0x2302bf0) allocated [ CSSD]2013-05-23 16:42:07.386 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:42:07.386 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302bf0 - 1,2,3 [ CSSD]2013-05-23 16:42:07.386 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x2302bf0) pid(25789) version 11:1:1:4 [ CSSD]2013-05-23 16:42:07.393 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2302bf0), client(1/0x23029c0) [ CSSD]2013-05-23 16:42:07.393 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:42:07.393 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23029c0 with con 0x2317960, requested num -1 [ CSSD]2013-05-23 16:42:07.393 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:07.393 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1759 (0x7f431bff5f90) [ CSSD]2013-05-23 16:42:07.393 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff5f90 (RPC#1759) tag(6df002c) sent to node 1 [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06df002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014744, gin 1, member 0 [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff5f90 (RPC#1759) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff5f90, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23029c0 [ CSSD]2013-05-23 16:42:07.394 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1759 [ CSSD]2013-05-23 16:42:07.401 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:42:07.401 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23029c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:07.401 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:07.401 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1760 (0x7f431bff6010) [ CSSD]2013-05-23 16:42:07.402 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6010 (RPC#1760) tag(6e0002c) sent to node 1 [ CSSD]2013-05-23 16:42:07.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06e0002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:07.402 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:07.402 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6010 (RPC#1760) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:07.402 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6010, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:07.402 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1760 [ CSSD]2013-05-23 16:42:07.402 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:07.604 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302bf0 [ CSSD]2013-05-23 16:42:07.604 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302bf0) con(0x231d6c0) skgpid ospid 25789 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:08.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:08.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:09.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:09.284 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:09.284 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:10.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:10.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:11.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:11.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:11.349 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014745, gin 1, member 0 [ CSSD]2013-05-23 16:42:11.349 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:42:11.349 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:42:11.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:11.355 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:42:11.355 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:42:12.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:12.300 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:13.247 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:14.285 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:42:14.285 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1761 (0x7f431bff6090) [ CSSD]2013-05-23 16:42:14.286 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6090 (RPC#1761) tag(6e1002c) sent to node 1 [ CSSD]2013-05-23 16:42:14.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:14.286 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1761 (0x7f431bff6090) [ CSSD]2013-05-23 16:42:14.286 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6090 (RPC#1761) state 6, flags 0x202 [ CSSD]2013-05-23 16:42:14.286 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6090, ret 0, client (nil) [ CSSD]2013-05-23 16:42:14.286 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1761 [ CSSD]2013-05-23 16:42:14.287 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:14.287 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:14.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:14.313 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:15.252 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:16.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:16.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:16.406 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 16:42:16.407 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:42:16.407 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 16:42:16.407 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d790) proc(0x22e01e0) pid(26069) version 11:1:1:4 [ CSSD]2013-05-23 16:42:16.424 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e01e0), client(1/0x23029c0) [ CSSD]2013-05-23 16:42:16.424 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:42:16.424 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23029c0 with con 0x22dada0, requested num -1 [ CSSD]2013-05-23 16:42:16.424 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:16.424 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1762 (0x7f431bff6110) [ CSSD]2013-05-23 16:42:16.424 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6110 (RPC#1762) tag(6e2002c) sent to node 1 [ CSSD]2013-05-23 16:42:16.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06e2002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:16.424 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014746, gin 1, member 0 [ CSSD]2013-05-23 16:42:16.424 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:42:16.424 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:16.425 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:42:16.425 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6110 (RPC#1762) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:16.425 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6110, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:16.425 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23029c0 [ CSSD]2013-05-23 16:42:16.425 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1762 [ CSSD]2013-05-23 16:42:16.431 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23029c0) [ CSSD]2013-05-23 16:42:16.431 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23029c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:16.431 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:16.432 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1763 (0x7f431bff6190) [ CSSD]2013-05-23 16:42:16.432 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6190 (RPC#1763) tag(6e3002c) sent to node 1 [ CSSD]2013-05-23 16:42:16.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06e3002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:16.432 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:16.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6190 (RPC#1763) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:16.432 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6190, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:16.432 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1763 [ CSSD]2013-05-23 16:42:16.432 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:16.635 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 16:42:16.635 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x231d790) skgpid ospid 26069 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:17.026 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 16:42:17.027 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:42:17.028 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 16:42:17.028 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1800) proc(0x22e01e0) pid(26110) version 11:1:1:4 [ CSSD]2013-05-23 16:42:17.147 [1191196992] >TRACE: clssgmAllocProc: (0x22daa40) allocated [ CSSD]2013-05-23 16:42:17.148 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:42:17.149 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22daa40 - 1,2,3 [ CSSD]2013-05-23 16:42:17.149 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317960) proc(0x22daa40) pid(26120) version 11:1:1:4 [ CSSD]2013-05-23 16:42:17.166 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:42:17.168 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:42:17.171 [1191196992] >TRACE: clssgmDeadProc: proc 0x22daa40 [ CSSD]2013-05-23 16:42:17.171 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22daa40) con(0x2317960) skgpid ospid 26120 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:17.249 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(71/0x23029c0) [ CSSD]2013-05-23 16:42:17.249 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 71 (0x23029c0) [ CSSD]2013-05-23 16:42:17.249 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x23029c0 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 16:42:17.249 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:42:17.249 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1764 (0x7f431bff6210) [ CSSD]2013-05-23 16:42:17.249 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6210 (RPC#1764) tag(6e4002c) sent to node 1 [ CSSD]2013-05-23 16:42:17.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06e4002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:17.249 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014747, gin 1, member 0 [ CSSD]2013-05-23 16:42:17.249 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x2302640/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:42:17.250 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:17.250 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:42:17.250 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6210 (RPC#1764) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:17.250 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6210, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:17.250 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23029c0 [ CSSD]2013-05-23 16:42:17.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1764 [ CSSD]2013-05-23 16:42:17.257 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 71 (0x23029c0) [ CSSD]2013-05-23 16:42:17.257 [1191196992] >TRACE: clssgmExitGrock: client 71 (0x23029c0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:42:17.257 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:42:17.257 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1765 (0x7f431bff6290) [ CSSD]2013-05-23 16:42:17.257 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6290 (RPC#1765) tag(6e5002c) sent to node 1 [ CSSD]2013-05-23 16:42:17.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06e5002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:17.257 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:42:17.258 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6290 (RPC#1765) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:17.258 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6290, ret 0, client 0x23029c0 member 0x22d7750 [ CSSD]2013-05-23 16:42:17.258 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:17.258 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1765 [ CSSD]2013-05-23 16:42:17.258 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:17.259 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 16:42:17.259 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22f1800) skgpid ospid 26110 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:18.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:18.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:19.270 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:19.290 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:19.290 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:20.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:20.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:21.017 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:21.017 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014748, gin 1, member 0 [ CSSD]2013-05-23 16:42:21.017 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:42:21.017 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:42:21.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:21.023 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:42:21.023 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:42:21.275 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:22.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:22.381 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:23.287 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:23.292 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:23.292 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:42:24.308 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:42:24.308 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1766 (0x7f431bff6310) [ CSSD]2013-05-23 16:42:24.309 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6310 (RPC#1766) tag(6e6002c) sent to node 1 [ CSSD]2013-05-23 16:42:24.309 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:24.309 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1766 (0x7f431bff6310) [ CSSD]2013-05-23 16:42:24.309 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6310 (RPC#1766) state 6, flags 0x202 [ CSSD]2013-05-23 16:42:24.309 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6310, ret 0, client (nil) [ CSSD]2013-05-23 16:42:24.309 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1766 [ CSSD]2013-05-23 16:42:24.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:24.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:24.850 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:42:24.851 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:42:24.852 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:42:24.852 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(26181) version 11:1:1:4 [ CSSD]2013-05-23 16:42:24.858 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:42:24.858 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:42:24.858 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:42:24.858 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:24.858 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1767 (0x7f431bff6390) [ CSSD]2013-05-23 16:42:24.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6390 (RPC#1767) tag(6e7002c) sent to node 1 [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06e7002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014749, gin 1, member 0 [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6390 (RPC#1767) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6390, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:42:24.859 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:42:24.860 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1767 [ CSSD]2013-05-23 16:42:24.867 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:42:24.867 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:24.867 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:24.867 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1768 (0x7f431bff6410) [ CSSD]2013-05-23 16:42:24.867 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6410 (RPC#1768) tag(6e8002c) sent to node 1 [ CSSD]2013-05-23 16:42:24.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06e8002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:24.868 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:24.868 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6410 (RPC#1768) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:24.868 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6410, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:42:24.868 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1768 [ CSSD]2013-05-23 16:42:24.868 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:25.070 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:42:25.070 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 26181 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:25.292 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:26.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:26.389 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:26.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:26.594 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014750, gin 1, member 0 [ CSSD]2013-05-23 16:42:26.594 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:42:26.594 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:42:26.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:26.601 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:42:26.601 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:42:27.317 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:28.295 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:28.295 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:28.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:28.394 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:29.370 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:30.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:30.405 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:31.375 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:31.663 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:42:31.663 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:42:31.664 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:42:31.664 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(26232) version 11:1:1:4 [ CSSD]2013-05-23 16:42:31.670 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:42:31.670 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:42:31.670 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c1214c0, requested num -1 [ CSSD]2013-05-23 16:42:31.670 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:31.670 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1769 (0x7f431bff6490) [ CSSD]2013-05-23 16:42:31.670 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6490 (RPC#1769) tag(6e9002c) sent to node 1 [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06e9002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014751, gin 1, member 0 [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6490 (RPC#1769) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6490, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:42:31.671 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1769 [ CSSD]2013-05-23 16:42:31.679 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:42:31.679 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:31.679 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:31.679 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1770 (0x7f431bff6510) [ CSSD]2013-05-23 16:42:31.679 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6510 (RPC#1770) tag(6ea002c) sent to node 1 [ CSSD]2013-05-23 16:42:31.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06ea002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:31.679 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:31.679 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6510 (RPC#1770) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:31.679 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6510, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:42:31.679 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1770 [ CSSD]2013-05-23 16:42:31.679 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:31.882 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:42:31.882 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 26232 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:32.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:32.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:33.298 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:33.298 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:33.388 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:34.332 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:42:34.333 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1771 (0x7f431bff6590) [ CSSD]2013-05-23 16:42:34.333 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6590 (RPC#1771) tag(6eb002c) sent to node 1 [ CSSD]2013-05-23 16:42:34.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:34.333 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1771 (0x7f431bff6590) [ CSSD]2013-05-23 16:42:34.333 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6590 (RPC#1771) state 6, flags 0x202 [ CSSD]2013-05-23 16:42:34.333 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6590, ret 0, client (nil) [ CSSD]2013-05-23 16:42:34.334 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1771 [ CSSD]2013-05-23 16:42:34.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:34.424 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:35.393 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:36.430 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:36.430 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:37.399 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:38.301 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:38.301 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:38.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:38.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:39.451 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:40.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:40.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:41.457 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:41.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:41.843 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014752, gin 1, member 0 [ CSSD]2013-05-23 16:42:41.843 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:42:41.843 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:42:41.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:41.850 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:42:41.851 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:42:42.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:42.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:43.304 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:43.305 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:43.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:44.355 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:42:44.355 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1772 (0x7f431bff6610) [ CSSD]2013-05-23 16:42:44.355 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6610 (RPC#1772) tag(6ec002c) sent to node 1 [ CSSD]2013-05-23 16:42:44.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:44.356 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1772 (0x7f431bff6610) [ CSSD]2013-05-23 16:42:44.356 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6610 (RPC#1772) state 6, flags 0x202 [ CSSD]2013-05-23 16:42:44.356 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6610, ret 0, client (nil) [ CSSD]2013-05-23 16:42:44.356 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1772 [ CSSD]2013-05-23 16:42:44.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:44.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:45.473 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:46.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:46.589 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:46.923 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:42:46.924 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:42:46.925 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:42:46.925 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(26429) version 11:1:1:4 [ CSSD]2013-05-23 16:42:46.931 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:42:46.931 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:42:46.931 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:42:46.931 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:46.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1773 (0x7f431bff6690) [ CSSD]2013-05-23 16:42:46.931 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6690 (RPC#1773) tag(6ed002c) sent to node 1 [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06ed002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014753, gin 1, member 0 [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6690 (RPC#1773) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6690, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:42:46.932 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1773 [ CSSD]2013-05-23 16:42:46.938 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:42:46.938 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:46.938 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:42:46.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1774 (0x7f431bff6710) [ CSSD]2013-05-23 16:42:46.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6710 (RPC#1774) tag(6ee002c) sent to node 1 [ CSSD]2013-05-23 16:42:46.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06ee002c) incarnation(174631051) [ CSSD]2013-05-23 16:42:46.939 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:42:46.939 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6710 (RPC#1774) state 6, flags 0x100 [ CSSD]2013-05-23 16:42:46.939 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6710, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:42:46.939 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1774 [ CSSD]2013-05-23 16:42:46.939 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:42:47.142 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:42:47.142 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 26429 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:42:47.307 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:47.307 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:42:47.477 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:48.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:48.613 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:49.489 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:50.706 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:50.706 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:51.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:52.310 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:52.310 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:52.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:52.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:53.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:54.382 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:42:54.382 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1775 (0x7f431bff6790) [ CSSD]2013-05-23 16:42:54.382 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6790 (RPC#1775) tag(6ef002c) sent to node 1 [ CSSD]2013-05-23 16:42:54.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:54.382 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1775 (0x7f431bff6790) [ CSSD]2013-05-23 16:42:54.382 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6790 (RPC#1775) state 6, flags 0x202 [ CSSD]2013-05-23 16:42:54.382 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6790, ret 0, client (nil) [ CSSD]2013-05-23 16:42:54.383 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1775 [ CSSD]2013-05-23 16:42:54.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:54.726 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:55.524 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:56.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:56.731 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:57.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:57.104 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014754, gin 1, member 0 [ CSSD]2013-05-23 16:42:57.104 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:42:57.104 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:42:57.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:57.110 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:42:57.110 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:42:57.313 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:42:57.313 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:42:57.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:42:58.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:42:58.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:42:59.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:00.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:00.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:01.588 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:02.173 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:43:02.174 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:43:02.175 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:43:02.175 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(26602) version 11:1:1:4 [ CSSD]2013-05-23 16:43:02.180 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:43:02.180 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:43:02.180 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c1214c0, requested num -1 [ CSSD]2013-05-23 16:43:02.180 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:02.181 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1776 (0x7f431bff6810) [ CSSD]2013-05-23 16:43:02.181 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6810 (RPC#1776) tag(6f0002c) sent to node 1 [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06f0002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014755, gin 1, member 0 [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6810 (RPC#1776) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6810, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:43:02.181 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:43:02.182 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1776 [ CSSD]2013-05-23 16:43:02.189 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:43:02.189 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:02.189 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:02.189 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1777 (0x7f431bff6890) [ CSSD]2013-05-23 16:43:02.189 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6890 (RPC#1777) tag(6f1002c) sent to node 1 [ CSSD]2013-05-23 16:43:02.189 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06f1002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:02.189 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:02.189 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6890 (RPC#1777) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:02.189 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6890, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:43:02.190 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1777 [ CSSD]2013-05-23 16:43:02.190 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:43:02.317 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:02.317 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:02.392 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:43:02.392 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 26602 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:43:02.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:02.813 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:03.678 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:04.406 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:43:04.407 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1778 (0x7f431bff6910) [ CSSD]2013-05-23 16:43:04.407 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6910 (RPC#1778) tag(6f2002c) sent to node 1 [ CSSD]2013-05-23 16:43:04.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:04.407 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1778 (0x7f431bff6910) [ CSSD]2013-05-23 16:43:04.407 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6910 (RPC#1778) state 6, flags 0x202 [ CSSD]2013-05-23 16:43:04.407 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6910, ret 0, client (nil) [ CSSD]2013-05-23 16:43:04.407 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1778 [ CSSD]2013-05-23 16:43:04.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:04.825 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:05.684 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:06.320 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:06.320 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:43:06.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:06.830 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:07.757 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:08.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:08.835 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:09.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:10.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:10.887 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:11.323 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:11.323 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:11.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:12.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:12.352 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014756, gin 1, member 0 [ CSSD]2013-05-23 16:43:12.352 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:43:12.352 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:43:12.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:12.360 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:43:12.360 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:43:12.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:12.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:13.860 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:14.434 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:43:14.434 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1779 (0x7f431bff6990) [ CSSD]2013-05-23 16:43:14.434 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6990 (RPC#1779) tag(6f3002c) sent to node 1 [ CSSD]2013-05-23 16:43:14.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:14.434 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1779 (0x7f431bff6990) [ CSSD]2013-05-23 16:43:14.434 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6990 (RPC#1779) state 6, flags 0x202 [ CSSD]2013-05-23 16:43:14.434 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6990, ret 0, client (nil) [ CSSD]2013-05-23 16:43:14.435 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1779 [ CSSD]2013-05-23 16:43:15.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:15.007 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:15.865 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:16.325 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:16.326 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:17.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:17.011 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:17.424 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:43:17.425 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:43:17.425 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:43:17.425 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(26773) version 11:1:1:4 [ CSSD]2013-05-23 16:43:17.431 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:43:17.431 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:43:17.431 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:43:17.431 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:17.431 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1780 (0x7f431bff6a10) [ CSSD]2013-05-23 16:43:17.431 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6a10 (RPC#1780) tag(6f4002c) sent to node 1 [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06f4002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014757, gin 1, member 0 [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6a10 (RPC#1780) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6a10, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:43:17.432 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1780 [ CSSD]2013-05-23 16:43:17.438 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:43:17.438 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:17.438 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:17.438 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1781 (0x7f431bff6a90) [ CSSD]2013-05-23 16:43:17.439 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6a90 (RPC#1781) tag(6f5002c) sent to node 1 [ CSSD]2013-05-23 16:43:17.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06f5002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:17.439 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:17.439 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6a90 (RPC#1781) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:17.439 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6a90, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:43:17.439 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1781 [ CSSD]2013-05-23 16:43:17.439 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:43:17.642 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:43:17.642 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 26773 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:43:17.877 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:19.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:19.016 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:19.902 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:21.027 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:21.027 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:21.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:21.273 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014758, gin 1, member 0 [ CSSD]2013-05-23 16:43:21.273 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:43:21.273 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:43:21.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:21.279 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:43:21.279 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:43:21.328 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:21.328 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:21.908 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:23.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:23.032 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:23.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:24.457 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:43:24.457 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1782 (0x7f431bff6b10) [ CSSD]2013-05-23 16:43:24.457 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6b10 (RPC#1782) tag(6f6002c) sent to node 1 [ CSSD]2013-05-23 16:43:24.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:24.458 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1782 (0x7f431bff6b10) [ CSSD]2013-05-23 16:43:24.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6b10 (RPC#1782) state 6, flags 0x202 [ CSSD]2013-05-23 16:43:24.458 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6b10, ret 0, client (nil) [ CSSD]2013-05-23 16:43:24.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1782 [ CSSD]2013-05-23 16:43:25.086 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:25.086 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:25.114 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:43:25.115 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:43:25.115 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:43:25.115 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(26865) version 11:1:1:4 [ CSSD]2013-05-23 16:43:25.132 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:43:25.132 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:43:25.132 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:43:25.132 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:25.132 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1783 (0x7f431bff6b90) [ CSSD]2013-05-23 16:43:25.132 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6b90 (RPC#1783) tag(6f7002c) sent to node 1 [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06f7002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014759, gin 1, member 0 [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6b90 (RPC#1783) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6b90, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:43:25.132 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:43:25.133 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1783 [ CSSD]2013-05-23 16:43:25.140 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:43:25.140 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:25.140 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:25.140 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1784 (0x7f431bff6c10) [ CSSD]2013-05-23 16:43:25.140 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6c10 (RPC#1784) tag(6f8002c) sent to node 1 [ CSSD]2013-05-23 16:43:25.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06f8002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:25.140 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:25.140 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6c10 (RPC#1784) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:25.140 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6c10, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:43:25.140 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1784 [ CSSD]2013-05-23 16:43:25.140 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:43:25.343 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:43:25.343 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 26865 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:43:25.966 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:26.331 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:26.331 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:27.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:27.091 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:27.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:27.600 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014760, gin 1, member 0 [ CSSD]2013-05-23 16:43:27.600 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:43:27.600 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:43:27.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:27.606 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:43:27.606 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:43:28.018 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:29.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:29.104 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:30.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:30.334 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:30.334 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:43:31.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:31.109 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:32.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:32.675 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:43:32.676 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:43:32.677 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:43:32.677 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(26922) version 11:1:1:4 [ CSSD]2013-05-23 16:43:32.683 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:43:32.683 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:43:32.683 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c1214c0, requested num -1 [ CSSD]2013-05-23 16:43:32.683 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:32.683 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1785 (0x7f431bff6c90) [ CSSD]2013-05-23 16:43:32.683 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6c90 (RPC#1785) tag(6f9002c) sent to node 1 [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06f9002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014761, gin 1, member 0 [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6c90 (RPC#1785) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6c90, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:43:32.684 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1785 [ CSSD]2013-05-23 16:43:32.690 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:43:32.690 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:32.690 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:32.690 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1786 (0x7f431bff6d10) [ CSSD]2013-05-23 16:43:32.691 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6d10 (RPC#1786) tag(6fa002c) sent to node 1 [ CSSD]2013-05-23 16:43:32.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06fa002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:32.691 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:32.691 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6d10 (RPC#1786) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:32.691 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6d10, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:43:32.691 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1786 [ CSSD]2013-05-23 16:43:32.691 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:43:32.893 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:43:32.893 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 26922 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:43:33.114 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:33.114 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:34.061 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:34.480 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:43:34.480 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1787 (0x7f431bff6d90) [ CSSD]2013-05-23 16:43:34.480 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6d90 (RPC#1787) tag(6fb002c) sent to node 1 [ CSSD]2013-05-23 16:43:34.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:34.481 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1787 (0x7f431bff6d90) [ CSSD]2013-05-23 16:43:34.481 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6d90 (RPC#1787) state 6, flags 0x202 [ CSSD]2013-05-23 16:43:34.481 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6d90, ret 0, client (nil) [ CSSD]2013-05-23 16:43:34.481 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1787 [ CSSD]2013-05-23 16:43:35.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:35.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:35.337 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:35.338 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:36.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:37.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:37.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:38.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:39.134 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:39.134 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:39.753 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:39.753 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014762, gin 1, member 0 [ CSSD]2013-05-23 16:43:39.753 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:43:39.753 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:43:39.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:39.759 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:43:39.759 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:43:39.853 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:43:39.854 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:43:39.856 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:43:39.856 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:43:39.858 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:43:39.859 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:43:40.104 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:40.341 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:40.341 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:41.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:41.140 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:42.109 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:42.839 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:42.839 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014763, gin 1, member 0 [ CSSD]2013-05-23 16:43:42.839 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:43:42.839 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:43:42.845 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:42.845 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:43:42.845 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:43:43.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:43.145 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:44.162 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:44.505 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:43:44.505 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1788 (0x7f431bff6e10) [ CSSD]2013-05-23 16:43:44.505 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6e10 (RPC#1788) tag(6fc002c) sent to node 1 [ CSSD]2013-05-23 16:43:44.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:44.506 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1788 (0x7f431bff6e10) [ CSSD]2013-05-23 16:43:44.506 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6e10 (RPC#1788) state 6, flags 0x202 [ CSSD]2013-05-23 16:43:44.506 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6e10, ret 0, client (nil) [ CSSD]2013-05-23 16:43:44.506 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1788 [ CSSD]2013-05-23 16:43:45.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:45.197 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:45.344 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:45.344 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:46.167 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:47.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:47.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:47.925 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:43:47.927 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:43:47.927 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:43:47.927 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(27064) version 11:1:1:4 [ CSSD]2013-05-23 16:43:47.933 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:43:47.933 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:43:47.933 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:43:47.933 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:47.933 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1789 (0x7f431bff6e90) [ CSSD]2013-05-23 16:43:47.933 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6e90 (RPC#1789) tag(6fd002c) sent to node 1 [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(06fd002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014764, gin 1, member 0 [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6e90 (RPC#1789) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff6e90, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:43:47.934 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1789 [ CSSD]2013-05-23 16:43:47.941 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:43:47.941 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:47.941 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:43:47.941 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1790 (0x7f431bff6f10) [ CSSD]2013-05-23 16:43:47.941 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6f10 (RPC#1790) tag(6fe002c) sent to node 1 [ CSSD]2013-05-23 16:43:47.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(06fe002c) incarnation(174631051) [ CSSD]2013-05-23 16:43:47.941 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:43:47.941 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6f10 (RPC#1790) state 6, flags 0x100 [ CSSD]2013-05-23 16:43:47.941 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff6f10, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:43:47.941 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1790 [ CSSD]2013-05-23 16:43:47.941 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:43:48.144 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:43:48.144 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 27064 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:43:48.178 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:49.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:49.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:49.347 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:49.347 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:43:50.204 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:51.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:51.220 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:52.210 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:53.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:53.225 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:54.037 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f2e60) [ CSSD]2013-05-23 16:43:54.037 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f2e60), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:43:54.037 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f2e60), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:43:54.037 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 16:43:54.048 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 16:43:54.048 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f3ef0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:43:54.048 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f3ef0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:43:54.048 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:43:54.048 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee920 [ CSSD]2013-05-23 16:43:54.048 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee920) con(0x22f4f40) skgpid 20007 ospid 20007 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:43:54.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:54.351 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:54.351 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:43:54.527 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:43:54.528 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1791 (0x7f431bff6f90) [ CSSD]2013-05-23 16:43:54.528 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff6f90 (RPC#1791) tag(6ff002c) sent to node 1 [ CSSD]2013-05-23 16:43:54.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:54.528 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1791 (0x7f431bff6f90) [ CSSD]2013-05-23 16:43:54.528 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff6f90 (RPC#1791) state 6, flags 0x202 [ CSSD]2013-05-23 16:43:54.528 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff6f90, ret 0, client (nil) [ CSSD]2013-05-23 16:43:54.529 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1791 [ CSSD]2013-05-23 16:43:55.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:55.237 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:56.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:57.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:57.242 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:58.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:58.087 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014765, gin 1, member 0 [ CSSD]2013-05-23 16:43:58.087 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:43:58.087 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:43:58.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:58.093 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:43:58.093 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:43:58.298 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:43:59.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:43:59.295 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:43:59.353 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:43:59.354 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:00.304 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:01.319 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:01.319 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:02.310 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:03.185 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:44:03.187 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:44:03.187 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:44:03.187 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(27216) version 11:1:1:4 [ CSSD]2013-05-23 16:44:03.203 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:44:03.204 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:44:03.204 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c129960, requested num -1 [ CSSD]2013-05-23 16:44:03.204 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:03.204 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1792 (0x7f431bff7010) [ CSSD]2013-05-23 16:44:03.204 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7010 (RPC#1792) tag(700002c) sent to node 1 [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0700002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014766, gin 1, member 0 [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7010 (RPC#1792) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7010, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:03.204 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:44:03.205 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1792 [ CSSD]2013-05-23 16:44:03.211 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:44:03.211 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:03.211 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:03.211 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1793 (0x7f431bff7090) [ CSSD]2013-05-23 16:44:03.211 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7090 (RPC#1793) tag(701002c) sent to node 1 [ CSSD]2013-05-23 16:44:03.212 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0701002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:03.212 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:03.212 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7090 (RPC#1793) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:03.212 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7090, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:03.212 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1793 [ CSSD]2013-05-23 16:44:03.212 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:44:03.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:03.324 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:03.414 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:44:03.414 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 27216 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:44:04.357 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:04.357 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:04.363 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:04.550 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:44:04.550 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1794 (0x7f431bff7110) [ CSSD]2013-05-23 16:44:04.550 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7110 (RPC#1794) tag(702002c) sent to node 1 [ CSSD]2013-05-23 16:44:04.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:04.551 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1794 (0x7f431bff7110) [ CSSD]2013-05-23 16:44:04.551 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7110 (RPC#1794) state 6, flags 0x202 [ CSSD]2013-05-23 16:44:04.551 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7110, ret 0, client (nil) [ CSSD]2013-05-23 16:44:04.551 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1794 [ CSSD]2013-05-23 16:44:05.337 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:05.337 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:06.388 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:07.342 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:07.342 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:08.359 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:08.360 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:44:08.400 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:09.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:09.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:10.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:11.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:11.401 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:12.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:13.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:13.346 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014767, gin 1, member 0 [ CSSD]2013-05-23 16:44:13.346 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:44:13.346 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:44:13.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:13.353 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:44:13.353 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:44:13.363 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:13.363 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:13.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:13.426 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:14.432 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:14.573 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:44:14.573 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1795 (0x7f431bff7190) [ CSSD]2013-05-23 16:44:14.573 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7190 (RPC#1795) tag(703002c) sent to node 1 [ CSSD]2013-05-23 16:44:14.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:14.574 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1795 (0x7f431bff7190) [ CSSD]2013-05-23 16:44:14.574 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7190 (RPC#1795) state 6, flags 0x202 [ CSSD]2013-05-23 16:44:14.574 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7190, ret 0, client (nil) [ CSSD]2013-05-23 16:44:14.574 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1795 [ CSSD]2013-05-23 16:44:15.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:15.456 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:16.438 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:17.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:17.461 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:18.365 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:18.365 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:18.456 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:44:18.457 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:44:18.457 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:44:18.457 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(27379) version 11:1:1:4 [ CSSD]2013-05-23 16:44:18.463 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:44:18.463 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:44:18.463 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:44:18.463 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:18.463 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1796 (0x7f431bff7210) [ CSSD]2013-05-23 16:44:18.463 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7210 (RPC#1796) tag(704002c) sent to node 1 [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0704002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014768, gin 1, member 0 [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7210 (RPC#1796) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7210, ret 0, client 0x7f431c091070 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:44:18.464 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1796 [ CSSD]2013-05-23 16:44:18.470 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:18.471 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:44:18.471 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:18.471 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:18.471 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1797 (0x7f431bff7290) [ CSSD]2013-05-23 16:44:18.471 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7290 (RPC#1797) tag(705002c) sent to node 1 [ CSSD]2013-05-23 16:44:18.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0705002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:18.472 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:18.472 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7290 (RPC#1797) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:18.472 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7290, ret 0, client 0x7f431c091070 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:18.472 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1797 [ CSSD]2013-05-23 16:44:18.472 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:44:18.675 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:44:18.675 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 27379 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:44:19.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:19.496 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:20.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:21.500 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:21.500 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:21.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:21.530 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014769, gin 1, member 0 [ CSSD]2013-05-23 16:44:21.530 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:44:21.530 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:44:21.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:21.535 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:44:21.535 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:44:22.500 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:23.368 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:23.368 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:23.505 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:23.505 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:24.531 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:24.596 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:44:24.596 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1798 (0x7f431bff7310) [ CSSD]2013-05-23 16:44:24.597 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7310 (RPC#1798) tag(706002c) sent to node 1 [ CSSD]2013-05-23 16:44:24.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:24.597 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1798 (0x7f431bff7310) [ CSSD]2013-05-23 16:44:24.597 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7310 (RPC#1798) state 6, flags 0x202 [ CSSD]2013-05-23 16:44:24.597 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7310, ret 0, client (nil) [ CSSD]2013-05-23 16:44:24.597 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1798 [ CSSD]2013-05-23 16:44:25.388 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:44:25.388 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:44:25.389 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:44:25.389 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(27465) version 11:1:1:4 [ CSSD]2013-05-23 16:44:25.395 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c0911f0) [ CSSD]2013-05-23 16:44:25.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0911f0) [ CSSD]2013-05-23 16:44:25.396 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0911f0 with con 0x7f431c090cf0, requested num -1 [ CSSD]2013-05-23 16:44:25.396 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:25.396 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1799 (0x7f431bff7390) [ CSSD]2013-05-23 16:44:25.396 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7390 (RPC#1799) tag(707002c) sent to node 1 [ CSSD]2013-05-23 16:44:25.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0707002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:25.396 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014770, gin 1, member 0 [ CSSD]2013-05-23 16:44:25.396 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091360/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:44:25.397 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:44:25.397 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:44:25.397 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7390 (RPC#1799) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:25.397 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7390, ret 0, client 0x7f431c0911f0 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:25.397 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0911f0 [ CSSD]2013-05-23 16:44:25.397 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1799 [ CSSD]2013-05-23 16:44:25.404 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0911f0) [ CSSD]2013-05-23 16:44:25.404 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0911f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:25.404 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:25.404 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1800 (0x7f431bff7410) [ CSSD]2013-05-23 16:44:25.404 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7410 (RPC#1800) tag(708002c) sent to node 1 [ CSSD]2013-05-23 16:44:25.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0708002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:25.405 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:25.405 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7410 (RPC#1800) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:25.405 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7410, ret 0, client 0x7f431c0911f0 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:25.405 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1800 [ CSSD]2013-05-23 16:44:25.405 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:44:25.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:25.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:25.608 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:44:25.608 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 27465 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:44:26.537 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:26.649 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:44:26.649 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:44:26.651 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:44:26.651 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:44:26.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:26.733 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014771, gin 1, member 0 [ CSSD]2013-05-23 16:44:26.733 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f75a0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:44:26.733 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:44:26.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:26.738 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:44:26.738 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:44:27.371 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:27.371 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:44:27.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:27.522 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:28.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:28.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:28.589 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014772, gin 1, member 0 [ CSSD]2013-05-23 16:44:28.589 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:44:28.589 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:44:28.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:28.595 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:44:28.595 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f1730) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:44:29.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:29.536 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:30.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:31.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:31.541 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:32.374 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:32.374 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:32.583 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:33.546 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:33.546 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:33.709 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:44:33.711 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:44:33.711 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:44:33.711 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(27529) version 11:1:1:4 [ CSSD]2013-05-23 16:44:33.717 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:44:33.717 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:44:33.717 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:44:33.717 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:33.717 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1801 (0x7f431bff7490) [ CSSD]2013-05-23 16:44:33.717 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7490 (RPC#1801) tag(709002c) sent to node 1 [ CSSD]2013-05-23 16:44:33.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0709002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:33.717 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014773, gin 1, member 0 [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7490 (RPC#1801) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7490, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:44:33.718 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1801 [ CSSD]2013-05-23 16:44:33.724 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:44:33.724 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:33.724 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:33.724 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1802 (0x7f431bff7510) [ CSSD]2013-05-23 16:44:33.724 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7510 (RPC#1802) tag(70a002c) sent to node 1 [ CSSD]2013-05-23 16:44:33.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(070a002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:33.725 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:33.725 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7510 (RPC#1802) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:33.725 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7510, ret 0, client 0x7f431c091070 member 0x7f431c134500 [ CSSD]2013-05-23 16:44:33.725 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1802 [ CSSD]2013-05-23 16:44:33.725 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:44:33.927 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:44:33.927 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 27529 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:44:34.593 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:34.621 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:44:34.621 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1803 (0x7f431bff7590) [ CSSD]2013-05-23 16:44:34.622 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7590 (RPC#1803) tag(70b002c) sent to node 1 [ CSSD]2013-05-23 16:44:34.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:34.622 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1803 (0x7f431bff7590) [ CSSD]2013-05-23 16:44:34.622 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7590 (RPC#1803) state 6, flags 0x202 [ CSSD]2013-05-23 16:44:34.622 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7590, ret 0, client (nil) [ CSSD]2013-05-23 16:44:34.622 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1803 [ CSSD]2013-05-23 16:44:35.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:35.599 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:36.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:37.377 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:37.377 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:37.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:37.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:38.671 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:39.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:39.616 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:40.697 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:41.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:41.621 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:42.380 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:42.381 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:42.789 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:43.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:43.627 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:43.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:43.837 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014774, gin 1, member 0 [ CSSD]2013-05-23 16:44:43.838 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:44:43.838 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:44:43.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:43.844 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:44:43.845 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:44:44.644 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:44:44.644 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1804 (0x7f431bff7610) [ CSSD]2013-05-23 16:44:44.644 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7610 (RPC#1804) tag(70c002c) sent to node 1 [ CSSD]2013-05-23 16:44:44.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:44.645 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1804 (0x7f431bff7610) [ CSSD]2013-05-23 16:44:44.645 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7610 (RPC#1804) state 6, flags 0x202 [ CSSD]2013-05-23 16:44:44.645 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7610, ret 0, client (nil) [ CSSD]2013-05-23 16:44:44.645 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1804 [ CSSD]2013-05-23 16:44:44.794 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:45.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:45.638 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:46.820 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:47.383 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:47.384 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:47.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:47.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:48.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:48.959 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:44:48.960 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:44:48.960 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:44:48.960 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(27680) version 11:1:1:4 [ CSSD]2013-05-23 16:44:48.966 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091140) [ CSSD]2013-05-23 16:44:48.966 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:44:48.966 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091140 with con 0x7f431c129b20, requested num -1 [ CSSD]2013-05-23 16:44:48.966 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:48.966 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1805 (0x7f431bff7690) [ CSSD]2013-05-23 16:44:48.967 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7690 (RPC#1805) tag(70d002c) sent to node 1 [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(070d002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014775, gin 1, member 0 [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0912b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7690 (RPC#1805) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7690, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:48.967 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091140 [ CSSD]2013-05-23 16:44:48.968 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1805 [ CSSD]2013-05-23 16:44:48.974 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091140) [ CSSD]2013-05-23 16:44:48.974 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091140), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:48.974 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:44:48.974 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1806 (0x7f431bff7710) [ CSSD]2013-05-23 16:44:48.974 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7710 (RPC#1806) tag(70e002c) sent to node 1 [ CSSD]2013-05-23 16:44:48.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(070e002c) incarnation(174631051) [ CSSD]2013-05-23 16:44:48.975 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:44:48.975 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7710 (RPC#1806) state 6, flags 0x100 [ CSSD]2013-05-23 16:44:48.975 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7710, ret 0, client 0x7f431c091140 member 0x7f431c134520 [ CSSD]2013-05-23 16:44:48.975 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1806 [ CSSD]2013-05-23 16:44:48.975 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:44:49.177 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:44:49.177 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 27680 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:44:49.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:49.647 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:50.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:51.386 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:51.386 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:44:51.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:51.652 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:52.890 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:53.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:53.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:54.667 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:44:54.667 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1807 (0x7f431bff7790) [ CSSD]2013-05-23 16:44:54.667 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7790 (RPC#1807) tag(70f002c) sent to node 1 [ CSSD]2013-05-23 16:44:54.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:54.668 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1807 (0x7f431bff7790) [ CSSD]2013-05-23 16:44:54.668 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7790 (RPC#1807) state 6, flags 0x202 [ CSSD]2013-05-23 16:44:54.668 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7790, ret 0, client (nil) [ CSSD]2013-05-23 16:44:54.668 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1807 [ CSSD]2013-05-23 16:44:54.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:55.709 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:55.709 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:56.389 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:44:56.389 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:44:56.921 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:57.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:57.714 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:44:58.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:44:59.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:59.085 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014776, gin 1, member 0 [ CSSD]2013-05-23 16:44:59.085 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:44:59.085 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:44:59.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:59.091 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:44:59.091 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:44:59.716 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:44:59.716 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:00.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:01.392 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:01.392 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:01.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:01.721 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:02.990 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:03.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:03.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:04.210 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:45:04.211 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:45:04.211 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:45:04.211 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c134630) pid(27817) version 11:1:1:4 [ CSSD]2013-05-23 16:45:04.217 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c091070) [ CSSD]2013-05-23 16:45:04.217 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:45:04.217 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091070 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:45:04.217 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:04.217 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1808 (0x7f431bff7810) [ CSSD]2013-05-23 16:45:04.217 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7810 (RPC#1808) tag(710002c) sent to node 1 [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0710002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014777, gin 1, member 0 [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0911e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7810 (RPC#1808) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7810, ret 0, client 0x7f431c091070 member 0x7f431c134520 [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091070 [ CSSD]2013-05-23 16:45:04.218 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1808 [ CSSD]2013-05-23 16:45:04.225 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091070) [ CSSD]2013-05-23 16:45:04.225 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:04.225 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:04.225 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1809 (0x7f431bff7890) [ CSSD]2013-05-23 16:45:04.225 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7890 (RPC#1809) tag(711002c) sent to node 1 [ CSSD]2013-05-23 16:45:04.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0711002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:04.225 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:04.225 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7890 (RPC#1809) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:04.225 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7890, ret 0, client 0x7f431c091070 member 0x7f431c134520 [ CSSD]2013-05-23 16:45:04.225 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1809 [ CSSD]2013-05-23 16:45:04.225 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:45:04.428 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:45:04.428 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c0840c0) skgpid ospid 27817 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:04.707 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:45:04.707 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1810 (0x7f431bff7910) [ CSSD]2013-05-23 16:45:04.708 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7910 (RPC#1810) tag(712002c) sent to node 1 [ CSSD]2013-05-23 16:45:04.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:04.708 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1810 (0x7f431bff7910) [ CSSD]2013-05-23 16:45:04.708 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7910 (RPC#1810) state 6, flags 0x202 [ CSSD]2013-05-23 16:45:04.708 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7910, ret 0, client (nil) [ CSSD]2013-05-23 16:45:04.708 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1810 [ CSSD]2013-05-23 16:45:05.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:05.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:05.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:06.395 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:06.395 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:07.022 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:07.845 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:07.845 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:08.349 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:45:08.349 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:45:08.350 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:45:08.350 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134630) pid(27905) version 11:1:1:4 [ CSSD]2013-05-23 16:45:08.892 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:45:08.892 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c13f020) skgpid ospid 27905 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:09.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:09.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:09.918 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:11.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:11.398 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:11.398 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:11.923 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:11.923 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:13.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:13.947 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:13.947 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:14.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:14.344 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014778, gin 1, member 0 [ CSSD]2013-05-23 16:45:14.344 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:45:14.344 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:45:14.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:14.351 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:45:14.351 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:45:14.736 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:45:14.736 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1811 (0x7f431bff7990) [ CSSD]2013-05-23 16:45:14.737 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7990 (RPC#1811) tag(713002c) sent to node 1 [ CSSD]2013-05-23 16:45:14.737 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:14.737 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1811 (0x7f431bff7990) [ CSSD]2013-05-23 16:45:14.737 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7990 (RPC#1811) state 6, flags 0x202 [ CSSD]2013-05-23 16:45:14.737 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7990, ret 0, client (nil) [ CSSD]2013-05-23 16:45:14.737 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1811 [ CSSD]2013-05-23 16:45:15.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:15.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:15.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:16.400 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:16.400 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:17.118 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:17.966 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:17.966 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:19.173 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:19.459 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134630) allocated [ CSSD]2013-05-23 16:45:19.460 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:45:19.461 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134630 - 1,2,3 [ CSSD]2013-05-23 16:45:19.461 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134da0) proc(0x7f431c134630) pid(28083) version 11:1:1:4 [ CSSD]2013-05-23 16:45:19.467 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134630), client(1/0x7f431c0911f0) [ CSSD]2013-05-23 16:45:19.467 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0911f0) [ CSSD]2013-05-23 16:45:19.467 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0911f0 with con 0x7f431c090cf0, requested num -1 [ CSSD]2013-05-23 16:45:19.467 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:19.467 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1812 (0x7f431bff7a10) [ CSSD]2013-05-23 16:45:19.467 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7a10 (RPC#1812) tag(714002c) sent to node 1 [ CSSD]2013-05-23 16:45:19.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0714002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:19.467 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014779, gin 1, member 0 [ CSSD]2013-05-23 16:45:19.467 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091360/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:45:19.468 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:45:19.468 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:45:19.468 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7a10 (RPC#1812) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:19.468 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7a10, ret 0, client 0x7f431c0911f0 member 0x7f431c134520 [ CSSD]2013-05-23 16:45:19.468 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0911f0 [ CSSD]2013-05-23 16:45:19.468 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1812 [ CSSD]2013-05-23 16:45:19.474 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0911f0) [ CSSD]2013-05-23 16:45:19.474 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0911f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:19.474 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:19.474 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1813 (0x7f431bff7a90) [ CSSD]2013-05-23 16:45:19.475 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7a90 (RPC#1813) tag(715002c) sent to node 1 [ CSSD]2013-05-23 16:45:19.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0715002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:19.475 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:19.475 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7a90 (RPC#1813) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:19.475 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7a90, ret 0, client 0x7f431c0911f0 member 0x7f431c134520 [ CSSD]2013-05-23 16:45:19.475 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1813 [ CSSD]2013-05-23 16:45:19.475 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:45:19.500 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22e91d0) [ CSSD]2013-05-23 16:45:19.500 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22e91d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:45:19.500 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22e91d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:45:19.500 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 16:45:19.505 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1d20) [ CSSD]2013-05-23 16:45:19.505 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1d20), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:45:19.505 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1d20), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:45:19.505 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:45:19.505 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 16:45:19.505 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c12f0c0) skgpid 22790 ospid 22790 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:19.678 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134630 [ CSSD]2013-05-23 16:45:19.678 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134630) con(0x7f431c134da0) skgpid ospid 28083 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:20.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:20.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:21.217 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:21.403 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:21.403 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:21.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:21.793 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014780, gin 1, member 0 [ CSSD]2013-05-23 16:45:21.793 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:45:21.793 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:45:21.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:21.801 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:45:21.801 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:45:22.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:22.043 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:23.271 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:24.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:24.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:24.760 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:45:24.760 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1814 (0x7f431bff7b10) [ CSSD]2013-05-23 16:45:24.760 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7b10 (RPC#1814) tag(716002c) sent to node 1 [ CSSD]2013-05-23 16:45:24.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:24.761 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1814 (0x7f431bff7b10) [ CSSD]2013-05-23 16:45:24.761 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7b10 (RPC#1814) state 6, flags 0x202 [ CSSD]2013-05-23 16:45:24.761 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7b10, ret 0, client (nil) [ CSSD]2013-05-23 16:45:24.761 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1814 [ CSSD]2013-05-23 16:45:25.277 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:25.649 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:45:25.650 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:45:25.651 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:45:25.651 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c129c20) pid(28165) version 11:1:1:4 [ CSSD]2013-05-23 16:45:25.657 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c129a80) [ CSSD]2013-05-23 16:45:25.657 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c129a80) [ CSSD]2013-05-23 16:45:25.657 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c129a80 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:45:25.657 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:25.657 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1815 (0x7f431bff7b90) [ CSSD]2013-05-23 16:45:25.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7b90 (RPC#1815) tag(717002c) sent to node 1 [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0717002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014781, gin 1, member 0 [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ad0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7b90 (RPC#1815) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7b90, ret 0, client 0x7f431c129a80 member 0x7f431c120dd0 [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c129a80 [ CSSD]2013-05-23 16:45:25.658 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1815 [ CSSD]2013-05-23 16:45:25.664 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c129a80) [ CSSD]2013-05-23 16:45:25.664 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c129a80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:25.664 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c120dd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:25.664 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1816 (0x7f431bff7c10) [ CSSD]2013-05-23 16:45:25.664 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7c10 (RPC#1816) tag(718002c) sent to node 1 [ CSSD]2013-05-23 16:45:25.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0718002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:25.665 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:25.665 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7c10 (RPC#1816) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:25.665 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7c10, ret 0, client 0x7f431c129a80 member 0x7f431c120dd0 [ CSSD]2013-05-23 16:45:25.665 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1816 [ CSSD]2013-05-23 16:45:25.665 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c120dd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:45:25.867 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:45:25.868 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c12f0c0) skgpid ospid 28165 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:26.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:26.081 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:26.406 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:26.406 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:27.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:28.086 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:28.086 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:29.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:29.593 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:29.593 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014782, gin 1, member 0 [ CSSD]2013-05-23 16:45:29.593 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:45:29.593 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:45:29.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:29.600 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:45:29.600 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:45:30.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:30.138 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:31.339 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:31.409 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:31.409 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:32.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:32.143 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:33.342 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:34.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:34.237 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:34.720 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:45:34.720 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:45:34.721 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:45:34.721 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c129c20) pid(28229) version 11:1:1:4 [ CSSD]2013-05-23 16:45:34.726 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c129a80) [ CSSD]2013-05-23 16:45:34.726 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c129a80) [ CSSD]2013-05-23 16:45:34.726 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c129a80 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:45:34.726 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:34.726 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1817 (0x7f431bff7c90) [ CSSD]2013-05-23 16:45:34.727 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7c90 (RPC#1817) tag(719002c) sent to node 1 [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0719002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014783, gin 1, member 0 [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ad0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7c90 (RPC#1817) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7c90, ret 0, client 0x7f431c129a80 member 0x7f431c120dd0 [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c129a80 [ CSSD]2013-05-23 16:45:34.727 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1817 [ CSSD]2013-05-23 16:45:34.734 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c129a80) [ CSSD]2013-05-23 16:45:34.734 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c129a80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:34.734 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c120dd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:34.734 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1818 (0x7f431bff7d10) [ CSSD]2013-05-23 16:45:34.734 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7d10 (RPC#1818) tag(71a002c) sent to node 1 [ CSSD]2013-05-23 16:45:34.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(071a002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:34.735 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:34.735 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7d10 (RPC#1818) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:34.735 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7d10, ret 0, client 0x7f431c129a80 member 0x7f431c120dd0 [ CSSD]2013-05-23 16:45:34.735 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1818 [ CSSD]2013-05-23 16:45:34.735 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c120dd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:45:34.784 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:45:34.785 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1819 (0x7f431bff7d90) [ CSSD]2013-05-23 16:45:34.785 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7d90 (RPC#1819) tag(71b002c) sent to node 1 [ CSSD]2013-05-23 16:45:34.785 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:34.785 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1819 (0x7f431bff7d90) [ CSSD]2013-05-23 16:45:34.785 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7d90 (RPC#1819) state 6, flags 0x202 [ CSSD]2013-05-23 16:45:34.785 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7d90, ret 0, client (nil) [ CSSD]2013-05-23 16:45:34.786 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1819 [ CSSD]2013-05-23 16:45:34.937 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:45:34.937 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c12f0c0) skgpid ospid 28229 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:35.347 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:36.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:36.260 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:36.412 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:36.412 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:37.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:38.265 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:38.265 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:39.404 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:40.319 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:40.319 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:41.410 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:41.415 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:41.415 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:42.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:42.324 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:43.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:44.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:44.336 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:44.809 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:45:44.809 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1820 (0x7f431bff7e10) [ CSSD]2013-05-23 16:45:44.809 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7e10 (RPC#1820) tag(71c002c) sent to node 1 [ CSSD]2013-05-23 16:45:44.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:44.810 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1820 (0x7f431bff7e10) [ CSSD]2013-05-23 16:45:44.810 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7e10 (RPC#1820) state 6, flags 0x202 [ CSSD]2013-05-23 16:45:44.810 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7e10, ret 0, client (nil) [ CSSD]2013-05-23 16:45:44.810 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1820 [ CSSD]2013-05-23 16:45:44.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:44.849 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014784, gin 1, member 0 [ CSSD]2013-05-23 16:45:44.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:45:44.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:45:44.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:44.855 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:45:44.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:45:45.427 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:46.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:46.340 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:46.418 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:46.418 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:47.007 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:45:47.007 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:45:47.008 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:45:47.008 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c129c20) pid(27153/27153) version 11:1:1:4 [ CSSD]2013-05-23 16:45:47.024 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c129900) [ CSSD]2013-05-23 16:45:47.024 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c129900) [ CSSD]2013-05-23 16:45:47.024 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:45:47.024 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:45:47.433 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:48.345 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:48.345 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:49.485 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:49.971 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090ef0) allocated [ CSSD]2013-05-23 16:45:49.972 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:45:49.972 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090ef0 - 1,2,3 [ CSSD]2013-05-23 16:45:49.972 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c129a70) proc(0x7f431c090ef0) pid(28393) version 11:1:1:4 [ CSSD]2013-05-23 16:45:49.979 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c090ef0), client(1/0x7f431c0915f0) [ CSSD]2013-05-23 16:45:49.979 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0915f0) [ CSSD]2013-05-23 16:45:49.979 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0915f0 with con 0x7f431c0910f0, requested num -1 [ CSSD]2013-05-23 16:45:49.979 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:49.979 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1821 (0x7f431bff7e90) [ CSSD]2013-05-23 16:45:49.979 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7e90 (RPC#1821) tag(71d002c) sent to node 1 [ CSSD]2013-05-23 16:45:49.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(071d002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:49.979 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014785, gin 1, member 0 [ CSSD]2013-05-23 16:45:49.979 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c091760/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:45:49.980 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:45:49.980 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:45:49.980 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7e90 (RPC#1821) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:49.980 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff7e90, ret 0, client 0x7f431c0915f0 member 0x7f431c090de0 [ CSSD]2013-05-23 16:45:49.980 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0915f0 [ CSSD]2013-05-23 16:45:49.980 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1821 [ CSSD]2013-05-23 16:45:49.986 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0915f0) [ CSSD]2013-05-23 16:45:49.986 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0915f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:49.986 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:45:49.986 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1822 (0x7f431bff7f10) [ CSSD]2013-05-23 16:45:49.986 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7f10 (RPC#1822) tag(71e002c) sent to node 1 [ CSSD]2013-05-23 16:45:49.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(071e002c) incarnation(174631051) [ CSSD]2013-05-23 16:45:49.986 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:45:49.986 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7f10 (RPC#1822) state 6, flags 0x100 [ CSSD]2013-05-23 16:45:49.986 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff7f10, ret 0, client 0x7f431c0915f0 member 0x7f431c090de0 [ CSSD]2013-05-23 16:45:49.987 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1822 [ CSSD]2013-05-23 16:45:49.987 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:45:50.189 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090ef0 [ CSSD]2013-05-23 16:45:50.189 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090ef0) con(0x7f431c129a70) skgpid ospid 28393 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:45:50.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:50.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:51.421 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:51.422 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:51.490 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:52.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:52.364 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:53.502 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:54.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:54.418 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:54.830 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:45:54.831 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1823 (0x7f431bff7f90) [ CSSD]2013-05-23 16:45:54.831 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff7f90 (RPC#1823) tag(71f002c) sent to node 1 [ CSSD]2013-05-23 16:45:54.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:54.831 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1823 (0x7f431bff7f90) [ CSSD]2013-05-23 16:45:54.831 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff7f90 (RPC#1823) state 6, flags 0x202 [ CSSD]2013-05-23 16:45:54.831 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff7f90, ret 0, client (nil) [ CSSD]2013-05-23 16:45:54.832 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1823 [ CSSD]2013-05-23 16:45:55.526 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:56.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:56.423 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:56.424 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:45:56.425 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:45:57.532 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:45:58.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:45:58.428 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:45:59.586 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:00.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:00.097 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014786, gin 1, member 0 [ CSSD]2013-05-23 16:46:00.097 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:46:00.097 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:46:00.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:00.104 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:46:00.104 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f19b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:46:00.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:00.439 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:01.428 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:01.428 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:01.592 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:02.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:02.443 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:03.602 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:04.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:04.458 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:04.854 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:46:04.855 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1824 (0x7f431bff8010) [ CSSD]2013-05-23 16:46:04.855 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8010 (RPC#1824) tag(720002c) sent to node 1 [ CSSD]2013-05-23 16:46:04.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:04.855 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1824 (0x7f431bff8010) [ CSSD]2013-05-23 16:46:04.855 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8010 (RPC#1824) state 6, flags 0x202 [ CSSD]2013-05-23 16:46:04.855 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8010, ret 0, client (nil) [ CSSD]2013-05-23 16:46:04.856 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1824 [ CSSD]2013-05-23 16:46:05.222 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c090ef0) allocated [ CSSD]2013-05-23 16:46:05.223 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:46:05.223 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c090ef0 - 1,2,3 [ CSSD]2013-05-23 16:46:05.223 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c090ef0) pid(28569) version 11:1:1:4 [ CSSD]2013-05-23 16:46:05.229 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c090ef0), client(1/0x7f431c121510) [ CSSD]2013-05-23 16:46:05.229 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:46:05.229 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121510 with con 0x7f431c091110, requested num -1 [ CSSD]2013-05-23 16:46:05.229 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:05.229 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1825 (0x7f431bff8090) [ CSSD]2013-05-23 16:46:05.229 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8090 (RPC#1825) tag(721002c) sent to node 1 [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0721002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014787, gin 1, member 0 [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0916e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8090 (RPC#1825) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8090, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121510 [ CSSD]2013-05-23 16:46:05.230 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1825 [ CSSD]2013-05-23 16:46:05.237 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121510) [ CSSD]2013-05-23 16:46:05.237 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:05.237 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:05.237 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1826 (0x7f431bff8110) [ CSSD]2013-05-23 16:46:05.237 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8110 (RPC#1826) tag(722002c) sent to node 1 [ CSSD]2013-05-23 16:46:05.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0722002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:05.237 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:05.237 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8110 (RPC#1826) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:05.237 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8110, ret 0, client 0x7f431c121510 member 0x7f431c090de0 [ CSSD]2013-05-23 16:46:05.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1826 [ CSSD]2013-05-23 16:46:05.238 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:46:05.440 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c090ef0 [ CSSD]2013-05-23 16:46:05.440 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c090ef0) con(0x7f431c12f0c0) skgpid ospid 28569 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:05.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:06.431 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:06.432 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:06.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:06.463 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:07.681 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:08.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:08.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:09.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:10.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:10.490 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:11.434 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:11.434 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:11.711 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:11.886 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 16:46:11.887 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:46:11.887 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 16:46:11.887 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c120ad0) proc(0x7f431c132e60) pid(28653/28653) version 11:1:1:4 [ CSSD]2013-05-23 16:46:11.903 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c132e60), client(1/0x7f431c133260) [ CSSD]2013-05-23 16:46:11.903 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c133260) [ CSSD]2013-05-23 16:46:11.903 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:46:11.903 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:46:12.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:12.495 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:12.838 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c132e60), client(2/0x7f431c133060) [ CSSD]2013-05-23 16:46:12.838 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c133060) [ CSSD]2013-05-23 16:46:12.838 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:46:12.838 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 16:46:13.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:14.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:14.528 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:14.879 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:46:14.879 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1827 (0x7f431bff8190) [ CSSD]2013-05-23 16:46:14.879 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8190 (RPC#1827) tag(723002c) sent to node 1 [ CSSD]2013-05-23 16:46:14.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:14.880 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1827 (0x7f431bff8190) [ CSSD]2013-05-23 16:46:14.880 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8190 (RPC#1827) state 6, flags 0x202 [ CSSD]2013-05-23 16:46:14.880 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8190, ret 0, client (nil) [ CSSD]2013-05-23 16:46:14.880 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1827 [ CSSD]2013-05-23 16:46:15.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:15.346 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014788, gin 1, member 0 [ CSSD]2013-05-23 16:46:15.346 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:46:15.346 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:46:15.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:15.353 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:46:15.353 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:46:15.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:16.438 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:16.438 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:16.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:16.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:17.781 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:18.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:18.578 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:18.821 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c133620) allocated [ CSSD]2013-05-23 16:46:18.822 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:46:18.822 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c133620 - 1,2,3 [ CSSD]2013-05-23 16:46:18.822 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c133e50) proc(0x7f431c133620) pid(28725) version 11:1:1:4 [ CSSD]2013-05-23 16:46:18.829 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c133620), client(1/0x7f431c137e70) [ CSSD]2013-05-23 16:46:18.829 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 16:46:18.829 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c137e70 with con 0x7f431c137b70, requested num -1 [ CSSD]2013-05-23 16:46:18.829 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:46:18.829 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1828 (0x7f431bff8210) [ CSSD]2013-05-23 16:46:18.829 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8210 (RPC#1828) tag(724002c) sent to node 1 [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0724002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014789, gin 1, member 0 [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082040/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8210 (RPC#1828) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8210, ret 0, client 0x7f431c137e70 member 0x7f431c137950 [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137e70 [ CSSD]2013-05-23 16:46:18.830 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1828 [ CSSD]2013-05-23 16:46:18.836 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 16:46:18.836 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137e70), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:46:18.836 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137950) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:46:18.836 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1829 (0x7f431bff8290) [ CSSD]2013-05-23 16:46:18.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8290 (RPC#1829) tag(725002c) sent to node 1 [ CSSD]2013-05-23 16:46:18.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0725002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:18.837 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:46:18.837 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8290 (RPC#1829) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:18.837 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8290, ret 0, client 0x7f431c137e70 member 0x7f431c137950 [ CSSD]2013-05-23 16:46:18.837 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1829 [ CSSD]2013-05-23 16:46:18.837 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c137950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:46:18.950 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083900) allocated [ CSSD]2013-05-23 16:46:18.951 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:46:18.952 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083900 - 1,2,3 [ CSSD]2013-05-23 16:46:18.952 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137d20) proc(0x7f431c083900) pid(28730) version 11:1:1:4 [ CSSD]2013-05-23 16:46:18.967 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:46:18.968 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:46:18.971 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083900 [ CSSD]2013-05-23 16:46:18.971 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083900) con(0x7f431c137d20) skgpid ospid 28730 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:19.039 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c133620 [ CSSD]2013-05-23 16:46:19.039 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c133620) con(0x7f431c133e50) skgpid ospid 28725 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:19.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:20.440 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:20.440 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:46:20.471 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137b30) allocated [ CSSD]2013-05-23 16:46:20.472 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:46:20.472 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137b30 - 1,2,3 [ CSSD]2013-05-23 16:46:20.472 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c133f60) proc(0x7f431c137b30) pid(28742) version 11:1:1:4 [ CSSD]2013-05-23 16:46:20.478 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c137b30), client(1/0x7f431c0821c0) [ CSSD]2013-05-23 16:46:20.478 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0821c0) [ CSSD]2013-05-23 16:46:20.478 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0821c0 with con 0x7f431c137de0, requested num -1 [ CSSD]2013-05-23 16:46:20.478 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:20.478 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1830 (0x7f431bff8310) [ CSSD]2013-05-23 16:46:20.478 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8310 (RPC#1830) tag(726002c) sent to node 1 [ CSSD]2013-05-23 16:46:20.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0726002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:20.478 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014790, gin 1, member 0 [ CSSD]2013-05-23 16:46:20.478 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082330/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:46:20.479 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:46:20.479 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:46:20.479 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8310 (RPC#1830) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:20.479 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8310, ret 0, client 0x7f431c0821c0 member 0x7f431c137a20 [ CSSD]2013-05-23 16:46:20.479 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0821c0 [ CSSD]2013-05-23 16:46:20.479 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1830 [ CSSD]2013-05-23 16:46:20.485 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0821c0) [ CSSD]2013-05-23 16:46:20.485 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0821c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:20.485 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137a20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:20.485 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1831 (0x7f431bff8390) [ CSSD]2013-05-23 16:46:20.485 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8390 (RPC#1831) tag(727002c) sent to node 1 [ CSSD]2013-05-23 16:46:20.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0727002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:20.486 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:20.486 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8390 (RPC#1831) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:20.486 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8390, ret 0, client 0x7f431c0821c0 member 0x7f431c137a20 [ CSSD]2013-05-23 16:46:20.486 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1831 [ CSSD]2013-05-23 16:46:20.486 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137a20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:46:20.631 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:20.631 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:20.688 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137b30 [ CSSD]2013-05-23 16:46:20.688 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137b30) con(0x7f431c133f60) skgpid ospid 28742 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:21.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:22.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:22.056 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014791, gin 1, member 0 [ CSSD]2013-05-23 16:46:22.056 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:46:22.056 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:46:22.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:22.062 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:46:22.062 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:46:22.655 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:22.655 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:23.804 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:24.147 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:24.147 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:46:24.147 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:46:24.147 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:46:24.147 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:46:24.147 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c11ab50) skgpid 23478 ospid 23478 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:24.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:24.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:24.905 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:46:24.905 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1832 (0x7f431bff8410) [ CSSD]2013-05-23 16:46:24.905 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8410 (RPC#1832) tag(728002c) sent to node 1 [ CSSD]2013-05-23 16:46:24.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:24.906 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1832 (0x7f431bff8410) [ CSSD]2013-05-23 16:46:24.906 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8410 (RPC#1832) state 6, flags 0x202 [ CSSD]2013-05-23 16:46:24.906 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8410, ret 0, client (nil) [ CSSD]2013-05-23 16:46:24.906 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1832 [ CSSD]2013-05-23 16:46:25.443 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:25.443 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:25.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:25.919 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:46:25.920 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:46:25.920 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:46:25.920 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c10a190) pid(28835) version 11:1:1:4 [ CSSD]2013-05-23 16:46:25.927 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c10a190), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:46:25.927 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:25.927 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c048500, requested num -1 [ CSSD]2013-05-23 16:46:25.927 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:25.927 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1833 (0x7f431bff8490) [ CSSD]2013-05-23 16:46:25.927 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8490 (RPC#1833) tag(729002c) sent to node 1 [ CSSD]2013-05-23 16:46:25.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0729002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:25.927 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014792, gin 1, member 0 [ CSSD]2013-05-23 16:46:25.927 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133af0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:46:25.928 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:46:25.928 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:46:25.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8490 (RPC#1833) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:25.928 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8490, ret 0, client 0x7f431c135120 member 0x7f431c133df0 [ CSSD]2013-05-23 16:46:25.928 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:46:25.928 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1833 [ CSSD]2013-05-23 16:46:25.934 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:25.934 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:25.934 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133df0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:25.934 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1834 (0x7f431bff8510) [ CSSD]2013-05-23 16:46:25.934 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8510 (RPC#1834) tag(72a002c) sent to node 1 [ CSSD]2013-05-23 16:46:25.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(072a002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:25.935 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:25.935 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8510 (RPC#1834) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:25.935 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8510, ret 0, client 0x7f431c135120 member 0x7f431c133df0 [ CSSD]2013-05-23 16:46:25.935 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1834 [ CSSD]2013-05-23 16:46:25.935 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133df0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:46:26.137 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:46:26.137 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c11ab50) skgpid ospid 28835 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:26.754 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:26.754 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:27.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:28.778 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:28.778 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:29.826 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:30.446 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:30.446 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:30.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:30.603 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014793, gin 1, member 0 [ CSSD]2013-05-23 16:46:30.603 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:46:30.603 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:46:30.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:30.610 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:46:30.610 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:46:30.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:30.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:31.850 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:32.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:32.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:33.884 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:34.928 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:46:34.929 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1835 (0x7f431bff8590) [ CSSD]2013-05-23 16:46:34.929 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8590 (RPC#1835) tag(72b002c) sent to node 1 [ CSSD]2013-05-23 16:46:34.929 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:34.929 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1835 (0x7f431bff8590) [ CSSD]2013-05-23 16:46:34.929 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8590 (RPC#1835) state 6, flags 0x202 [ CSSD]2013-05-23 16:46:34.929 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8590, ret 0, client (nil) [ CSSD]2013-05-23 16:46:34.930 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1835 [ CSSD]2013-05-23 16:46:34.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:34.949 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:35.449 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:35.450 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:35.719 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:46:35.720 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:46:35.721 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:46:35.721 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c10a190) pid(28916) version 11:1:1:4 [ CSSD]2013-05-23 16:46:35.727 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c10a190), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:46:35.727 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:35.727 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 16:46:35.727 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:35.727 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1836 (0x7f431bff8610) [ CSSD]2013-05-23 16:46:35.727 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8610 (RPC#1836) tag(72c002c) sent to node 1 [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(072c002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014794, gin 1, member 0 [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133a20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8610 (RPC#1836) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8610, ret 0, client 0x7f431c135120 member 0x7f431c133d20 [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:46:35.728 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1836 [ CSSD]2013-05-23 16:46:35.734 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:35.734 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:35.734 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133d20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:35.734 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1837 (0x7f431bff8690) [ CSSD]2013-05-23 16:46:35.734 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8690 (RPC#1837) tag(72d002c) sent to node 1 [ CSSD]2013-05-23 16:46:35.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(072d002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:35.735 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:35.735 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8690 (RPC#1837) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:35.735 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8690, ret 0, client 0x7f431c135120 member 0x7f431c133d20 [ CSSD]2013-05-23 16:46:35.735 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1837 [ CSSD]2013-05-23 16:46:35.735 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133d20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:46:35.910 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:35.937 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:46:35.937 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c12a550) skgpid ospid 28916 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:36.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:36.974 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:37.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:38.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:38.980 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:39.452 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:39.452 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:46:39.929 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:41.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:41.071 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:41.935 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:43.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:43.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:44.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:44.455 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:44.456 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:44.952 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:46:44.952 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1838 (0x7f431bff8710) [ CSSD]2013-05-23 16:46:44.953 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8710 (RPC#1838) tag(72e002c) sent to node 1 [ CSSD]2013-05-23 16:46:44.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:44.953 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1838 (0x7f431bff8710) [ CSSD]2013-05-23 16:46:44.953 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8710 (RPC#1838) state 6, flags 0x202 [ CSSD]2013-05-23 16:46:44.953 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8710, ret 0, client (nil) [ CSSD]2013-05-23 16:46:44.953 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1838 [ CSSD]2013-05-23 16:46:45.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:45.130 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:45.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:45.853 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014795, gin 1, member 0 [ CSSD]2013-05-23 16:46:45.853 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:46:45.853 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:46:45.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:45.859 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:46:45.859 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f19b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:46:46.011 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:47.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:47.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:48.022 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:49.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:49.148 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:49.459 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:49.459 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:50.028 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:50.980 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 16:46:50.981 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:46:50.981 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 16:46:50.981 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c10a190) pid(29108) version 11:1:1:4 [ CSSD]2013-05-23 16:46:50.998 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c10a190), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:46:50.998 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:50.998 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c048500, requested num -1 [ CSSD]2013-05-23 16:46:50.998 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:50.998 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1839 (0x7f431bff8790) [ CSSD]2013-05-23 16:46:50.998 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8790 (RPC#1839) tag(72f002c) sent to node 1 [ CSSD]2013-05-23 16:46:50.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(072f002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:50.998 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014796, gin 1, member 0 [ CSSD]2013-05-23 16:46:50.998 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133af0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:46:50.999 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:46:50.999 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:46:50.999 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8790 (RPC#1839) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:50.999 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8790, ret 0, client 0x7f431c135120 member 0x7f431c133df0 [ CSSD]2013-05-23 16:46:50.999 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:46:50.999 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1839 [ CSSD]2013-05-23 16:46:51.006 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:51.006 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:51.006 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c133df0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:46:51.006 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1840 (0x7f431bff8810) [ CSSD]2013-05-23 16:46:51.006 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8810 (RPC#1840) tag(730002c) sent to node 1 [ CSSD]2013-05-23 16:46:51.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0730002c) incarnation(174631051) [ CSSD]2013-05-23 16:46:51.007 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:46:51.007 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8810 (RPC#1840) state 6, flags 0x100 [ CSSD]2013-05-23 16:46:51.007 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8810, ret 0, client 0x7f431c135120 member 0x7f431c133df0 [ CSSD]2013-05-23 16:46:51.007 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1840 [ CSSD]2013-05-23 16:46:51.007 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c133df0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:46:51.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:51.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:51.209 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 16:46:51.209 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c12a550) skgpid ospid 29108 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:46:52.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:53.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:53.158 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:54.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:54.462 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:54.462 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:46:54.977 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:46:54.978 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1841 (0x7f431bff8890) [ CSSD]2013-05-23 16:46:54.978 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8890 (RPC#1841) tag(731002c) sent to node 1 [ CSSD]2013-05-23 16:46:54.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:54.978 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1841 (0x7f431bff8890) [ CSSD]2013-05-23 16:46:54.978 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8890 (RPC#1841) state 6, flags 0x202 [ CSSD]2013-05-23 16:46:54.978 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8890, ret 0, client (nil) [ CSSD]2013-05-23 16:46:54.978 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1841 [ CSSD]2013-05-23 16:46:55.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:55.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:56.101 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:56.493 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 16:46:56.494 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:46:56.494 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 16:46:56.494 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109eb0) proc(0x7f431c108370) pid(28819/28819) version 11:1:1:4 [ CSSD]2013-05-23 16:46:56.511 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c108370), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:46:56.511 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:46:56.511 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:46:56.511 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:46:57.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:57.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:46:58.113 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:46:58.465 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:46:58.465 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:46:59.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:46:59.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:00.119 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:01.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:01.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014797, gin 1, member 0 [ CSSD]2013-05-23 16:47:01.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:01.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:01.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:01.107 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:01.107 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x231d6c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:01.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:01.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:02.125 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:03.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:03.237 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:03.467 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:03.468 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:04.135 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:05.012 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:47:05.013 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1842 (0x7f431bff8910) [ CSSD]2013-05-23 16:47:05.013 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8910 (RPC#1842) tag(732002c) sent to node 1 [ CSSD]2013-05-23 16:47:05.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:05.013 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1842 (0x7f431bff8910) [ CSSD]2013-05-23 16:47:05.013 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8910 (RPC#1842) state 6, flags 0x202 [ CSSD]2013-05-23 16:47:05.013 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8910, ret 0, client (nil) [ CSSD]2013-05-23 16:47:05.014 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1842 [ CSSD]2013-05-23 16:47:05.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:05.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:06.141 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:06.251 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137bb0) allocated [ CSSD]2013-05-23 16:47:06.252 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:47:06.253 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137bb0 - 1,2,3 [ CSSD]2013-05-23 16:47:06.253 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137bb0) pid(29277) version 11:1:1:4 [ CSSD]2013-05-23 16:47:06.258 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c137bb0), client(1/0x7f431c0820f0) [ CSSD]2013-05-23 16:47:06.258 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0820f0) [ CSSD]2013-05-23 16:47:06.258 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0820f0 with con 0x7f431c1347b0, requested num -1 [ CSSD]2013-05-23 16:47:06.258 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:06.258 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1843 (0x7f431bff8990) [ CSSD]2013-05-23 16:47:06.258 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8990 (RPC#1843) tag(733002c) sent to node 1 [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0733002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014798, gin 1, member 0 [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082260/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8990 (RPC#1843) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8990, ret 0, client 0x7f431c0820f0 member 0x7f431c082560 [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0820f0 [ CSSD]2013-05-23 16:47:06.259 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1843 [ CSSD]2013-05-23 16:47:06.265 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0820f0) [ CSSD]2013-05-23 16:47:06.266 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0820f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:06.266 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082560) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:06.266 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1844 (0x7f431bff8a10) [ CSSD]2013-05-23 16:47:06.266 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8a10 (RPC#1844) tag(734002c) sent to node 1 [ CSSD]2013-05-23 16:47:06.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0734002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:06.266 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:06.266 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8a10 (RPC#1844) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:06.266 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8a10, ret 0, client 0x7f431c0820f0 member 0x7f431c082560 [ CSSD]2013-05-23 16:47:06.267 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1844 [ CSSD]2013-05-23 16:47:06.267 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c082560) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:47:06.469 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137bb0 [ CSSD]2013-05-23 16:47:06.469 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137bb0) con(0x7f431c11ab50) skgpid ospid 29277 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:07.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:07.256 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:08.194 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:08.471 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:08.471 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:09.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:09.248 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:10.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:11.032 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137bb0) allocated [ CSSD]2013-05-23 16:47:11.033 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:47:11.034 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137bb0 - 1,2,3 [ CSSD]2013-05-23 16:47:11.034 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c137bb0) pid(28124/28124) version 11:1:1:4 [ CSSD]2013-05-23 16:47:11.050 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c137bb0), client(1/0x7f431c081f70) [ CSSD]2013-05-23 16:47:11.050 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c081f70) [ CSSD]2013-05-23 16:47:11.050 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:47:11.050 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 16:47:11.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:11.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:12.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:12.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:12.910 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014799, gin 1, member 0 [ CSSD]2013-05-23 16:47:12.910 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:12.910 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:12.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:12.917 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:12.917 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:13.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:13.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:13.474 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:13.474 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:14.296 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:15.037 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:47:15.038 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1845 (0x7f431bff8a90) [ CSSD]2013-05-23 16:47:15.038 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8a90 (RPC#1845) tag(735002c) sent to node 1 [ CSSD]2013-05-23 16:47:15.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:15.038 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1845 (0x7f431bff8a90) [ CSSD]2013-05-23 16:47:15.038 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8a90 (RPC#1845) state 6, flags 0x202 [ CSSD]2013-05-23 16:47:15.038 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8a90, ret 0, client (nil) [ CSSD]2013-05-23 16:47:15.038 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1845 [ CSSD]2013-05-23 16:47:15.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:15.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:16.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:16.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:16.349 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014800, gin 1, member 0 [ CSSD]2013-05-23 16:47:16.349 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:16.349 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:16.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:16.355 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:16.355 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:17.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:17.336 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:17.341 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ad20) allocated [ CSSD]2013-05-23 16:47:17.342 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:47:17.342 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ad20 - 1,2,3 [ CSSD]2013-05-23 16:47:17.342 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c13ad20) pid(29592) version 11:1:1:4 [ CSSD]2013-05-23 16:47:17.473 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c14fb20) allocated [ CSSD]2013-05-23 16:47:17.474 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:47:17.474 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c14fb20 - 1,2,3 [ CSSD]2013-05-23 16:47:17.474 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13b530) proc(0x7f431c14fb20) pid(29602) version 11:1:1:4 [ CSSD]2013-05-23 16:47:17.476 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:17.477 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:47:17.500 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:47:17.502 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:47:17.504 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c14fb20 [ CSSD]2013-05-23 16:47:17.504 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c14fb20) con(0x7f431c13b530) skgpid ospid 29602 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:17.573 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(72/0x7f431c13b4b0) [ CSSD]2013-05-23 16:47:17.573 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 72 (0x7f431c13b4b0) [ CSSD]2013-05-23 16:47:17.573 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c13b4b0 with con 0x7f431c13afa0, requested num -1 [ CSSD]2013-05-23 16:47:17.573 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:47:17.573 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1846 (0x7f431bff8b10) [ CSSD]2013-05-23 16:47:17.574 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8b10 (RPC#1846) tag(736002c) sent to node 1 [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0736002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014801, gin 1, member 0 [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c13b620/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8b10 (RPC#1846) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8b10, ret 0, client 0x7f431c13b4b0 member 0x7f431c13ac00 [ CSSD]2013-05-23 16:47:17.574 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13b4b0 [ CSSD]2013-05-23 16:47:17.575 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1846 [ CSSD]2013-05-23 16:47:17.581 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 72 (0x7f431c13b4b0) [ CSSD]2013-05-23 16:47:17.581 [1191196992] >TRACE: clssgmExitGrock: client 72 (0x7f431c13b4b0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:47:17.581 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ac00) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:47:17.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1847 (0x7f431bff8b90) [ CSSD]2013-05-23 16:47:17.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8b90 (RPC#1847) tag(737002c) sent to node 1 [ CSSD]2013-05-23 16:47:17.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0737002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:17.582 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:47:17.582 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8b90 (RPC#1847) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:17.582 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8b90, ret 0, client 0x7f431c13b4b0 member 0x7f431c13ac00 [ CSSD]2013-05-23 16:47:17.582 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1847 [ CSSD]2013-05-23 16:47:17.582 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c13ac00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:47:17.583 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ad20 [ CSSD]2013-05-23 16:47:17.583 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ad20) con(0x7f431c12a550) skgpid ospid 29592 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:18.315 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:19.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:19.348 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:20.321 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:21.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:21.352 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:21.504 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 16:47:21.505 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:47:21.506 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 16:47:21.506 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1d20) proc(0x22e01e0) pid(29636) version 11:1:1:4 [ CSSD]2013-05-23 16:47:21.522 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22dabc0) [ CSSD]2013-05-23 16:47:21.522 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 16:47:21.522 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabc0 with con 0x22dada0, requested num -1 [ CSSD]2013-05-23 16:47:21.522 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:21.522 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1848 (0x7f431bff8c10) [ CSSD]2013-05-23 16:47:21.522 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8c10 (RPC#1848) tag(738002c) sent to node 1 [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0738002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014802, gin 1, member 0 [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8c10 (RPC#1848) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8c10, ret 0, client 0x22dabc0 member 0x22d7750 [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 16:47:21.523 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1848 [ CSSD]2013-05-23 16:47:21.530 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 16:47:21.530 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:21.530 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:21.530 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1849 (0x7f431bff8c90) [ CSSD]2013-05-23 16:47:21.530 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8c90 (RPC#1849) tag(739002c) sent to node 1 [ CSSD]2013-05-23 16:47:21.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0739002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:21.531 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:21.531 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8c90 (RPC#1849) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:21.531 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8c90, ret 0, client 0x22dabc0 member 0x22d7750 [ CSSD]2013-05-23 16:47:21.531 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1849 [ CSSD]2013-05-23 16:47:21.531 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:47:21.733 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 16:47:21.733 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22f1d20) skgpid ospid 29636 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:22.310 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:22.310 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014803, gin 1, member 0 [ CSSD]2013-05-23 16:47:22.310 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:22.310 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:22.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:22.317 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:22.317 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:22.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:22.479 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:22.480 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:23.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:23.357 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:24.417 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:25.063 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:47:25.063 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1850 (0x7f431bff8d10) [ CSSD]2013-05-23 16:47:25.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8d10 (RPC#1850) tag(73a002c) sent to node 1 [ CSSD]2013-05-23 16:47:25.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:25.064 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1850 (0x7f431bff8d10) [ CSSD]2013-05-23 16:47:25.064 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8d10 (RPC#1850) state 6, flags 0x202 [ CSSD]2013-05-23 16:47:25.064 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8d10, ret 0, client (nil) [ CSSD]2013-05-23 16:47:25.064 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1850 [ CSSD]2013-05-23 16:47:25.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:25.372 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:26.185 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 16:47:26.186 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:47:26.187 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 16:47:26.187 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1d20) proc(0x22e01e0) pid(29704) version 11:1:1:4 [ CSSD]2013-05-23 16:47:26.193 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e01e0), client(1/0x22dabc0) [ CSSD]2013-05-23 16:47:26.193 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 16:47:26.193 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabc0 with con 0x231d790, requested num -1 [ CSSD]2013-05-23 16:47:26.193 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:26.193 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1851 (0x7f431bff8d90) [ CSSD]2013-05-23 16:47:26.193 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8d90 (RPC#1851) tag(73b002c) sent to node 1 [ CSSD]2013-05-23 16:47:26.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(073b002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:26.193 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014804, gin 1, member 0 [ CSSD]2013-05-23 16:47:26.193 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:47:26.194 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:47:26.194 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:47:26.194 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8d90 (RPC#1851) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:26.194 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8d90, ret 0, client 0x22dabc0 member 0x22d7750 [ CSSD]2013-05-23 16:47:26.194 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 16:47:26.194 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1851 [ CSSD]2013-05-23 16:47:26.200 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 16:47:26.200 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:26.200 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:26.200 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1852 (0x7f431bff8e10) [ CSSD]2013-05-23 16:47:26.201 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8e10 (RPC#1852) tag(73c002c) sent to node 1 [ CSSD]2013-05-23 16:47:26.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(073c002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:26.201 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:26.201 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8e10 (RPC#1852) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:26.201 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8e10, ret 0, client 0x22dabc0 member 0x22d7750 [ CSSD]2013-05-23 16:47:26.201 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1852 [ CSSD]2013-05-23 16:47:26.201 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:47:26.404 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 16:47:26.404 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22f1d20) skgpid ospid 29704 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:26.422 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:27.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:27.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:27.482 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:27.483 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:28.435 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:29.379 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:29.379 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:30.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:31.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:31.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:31.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:31.598 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014805, gin 1, member 0 [ CSSD]2013-05-23 16:47:31.598 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:31.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:31.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:31.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:31.605 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:32.446 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:32.486 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:32.486 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:33.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:33.389 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:34.497 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:35.087 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:47:35.088 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1853 (0x7f431bff8e90) [ CSSD]2013-05-23 16:47:35.088 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8e90 (RPC#1853) tag(73d002c) sent to node 1 [ CSSD]2013-05-23 16:47:35.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:35.089 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1853 (0x7f431bff8e90) [ CSSD]2013-05-23 16:47:35.089 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8e90 (RPC#1853) state 6, flags 0x202 [ CSSD]2013-05-23 16:47:35.089 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff8e90, ret 0, client (nil) [ CSSD]2013-05-23 16:47:35.089 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1853 [ CSSD]2013-05-23 16:47:35.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:35.442 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:36.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:36.766 [1191196992] >TRACE: clssgmAllocProc: (0x22f1730) allocated [ CSSD]2013-05-23 16:47:36.767 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:47:36.768 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f1730 - 1,2,3 [ CSSD]2013-05-23 16:47:36.768 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x22f1730) pid(29801) version 11:1:1:4 [ CSSD]2013-05-23 16:47:36.773 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22f1730), client(1/0x22dabd0) [ CSSD]2013-05-23 16:47:36.773 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabd0) [ CSSD]2013-05-23 16:47:36.773 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabd0 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 16:47:36.773 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:36.773 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1854 (0x7f431bff8f10) [ CSSD]2013-05-23 16:47:36.773 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8f10 (RPC#1854) tag(73e002c) sent to node 1 [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(073e002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014806, gin 1, member 0 [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8f10 (RPC#1854) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff8f10, ret 0, client 0x22dabd0 member 0x22d7820 [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabd0 [ CSSD]2013-05-23 16:47:36.774 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1854 [ CSSD]2013-05-23 16:47:36.780 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabd0) [ CSSD]2013-05-23 16:47:36.780 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:36.780 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7820) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:36.780 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1855 (0x7f431bff8f90) [ CSSD]2013-05-23 16:47:36.781 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff8f90 (RPC#1855) tag(73f002c) sent to node 1 [ CSSD]2013-05-23 16:47:36.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(073f002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:36.781 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:36.781 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff8f90 (RPC#1855) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:36.781 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff8f90, ret 0, client 0x22dabd0 member 0x22d7820 [ CSSD]2013-05-23 16:47:36.781 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1855 [ CSSD]2013-05-23 16:47:36.781 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7820) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:47:36.984 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f1730 [ CSSD]2013-05-23 16:47:36.984 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f1730) con(0x231d6c0) skgpid ospid 29801 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:37.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:37.446 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:37.489 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:37.489 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:38.515 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:39.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:39.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:40.540 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:41.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:41.465 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:41.492 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:41.492 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:47:42.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:42.437 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014807, gin 1, member 0 [ CSSD]2013-05-23 16:47:42.437 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:42.437 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:42.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:42.443 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:42.443 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:42.545 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:43.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:43.470 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:44.637 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:45.117 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:47:45.117 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1856 (0x7f431bff9010) [ CSSD]2013-05-23 16:47:45.118 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9010 (RPC#1856) tag(740002c) sent to node 1 [ CSSD]2013-05-23 16:47:45.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:45.118 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1856 (0x7f431bff9010) [ CSSD]2013-05-23 16:47:45.118 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9010 (RPC#1856) state 6, flags 0x202 [ CSSD]2013-05-23 16:47:45.118 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9010, ret 0, client (nil) [ CSSD]2013-05-23 16:47:45.118 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1856 [ CSSD]2013-05-23 16:47:45.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:45.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:46.495 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:46.496 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:46.641 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:46.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:46.854 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014808, gin 1, member 0 [ CSSD]2013-05-23 16:47:46.854 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:47:46.854 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:47:46.861 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:46.861 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:47:46.861 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f19b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:47:47.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:47.566 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:48.715 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:49.580 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:49.580 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:49.604 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:47:49.604 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0e6e30), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:47:49.604 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0e6e30), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:47:49.604 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 16:47:49.609 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:47:49.609 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:47:49.609 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:47:49.609 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 16:47:49.609 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136070 [ CSSD]2013-05-23 16:47:49.609 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136070) con(0x7f431c108e60) skgpid 24643 ospid 24643 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:49.614 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 16:47:49.614 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:47:49.614 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0cf720), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:47:49.614 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 16:47:49.614 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1246d0 [ CSSD]2013-05-23 16:47:49.614 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1246d0) con(0x7f431c109f70) skgpid 24641 ospid 24641 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:50.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:51.498 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:51.498 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:51.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:51.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:52.016 [1191196992] >TRACE: clssgmAllocProc: (0x22e01e0) allocated [ CSSD]2013-05-23 16:47:52.017 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:47:52.017 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e01e0 - 1,2,3 [ CSSD]2013-05-23 16:47:52.017 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1d20) proc(0x22e01e0) pid(30005) version 11:1:1:4 [ CSSD]2013-05-23 16:47:52.023 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e01e0), client(1/0x22dabc0) [ CSSD]2013-05-23 16:47:52.023 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 16:47:52.023 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabc0 with con 0x22f1a80, requested num -1 [ CSSD]2013-05-23 16:47:52.023 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:52.023 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1857 (0x7f431bff9090) [ CSSD]2013-05-23 16:47:52.024 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9090 (RPC#1857) tag(741002c) sent to node 1 [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0741002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014809, gin 1, member 0 [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9090 (RPC#1857) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9090, ret 0, client 0x22dabc0 member 0x22d7750 [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 16:47:52.024 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1857 [ CSSD]2013-05-23 16:47:52.031 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 16:47:52.031 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:52.031 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:47:52.031 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1858 (0x7f431bff9110) [ CSSD]2013-05-23 16:47:52.031 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9110 (RPC#1858) tag(742002c) sent to node 1 [ CSSD]2013-05-23 16:47:52.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0742002c) incarnation(174631051) [ CSSD]2013-05-23 16:47:52.032 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:47:52.032 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9110 (RPC#1858) state 6, flags 0x100 [ CSSD]2013-05-23 16:47:52.032 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9110, ret 0, client 0x22dabc0 member 0x22d7750 [ CSSD]2013-05-23 16:47:52.032 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1858 [ CSSD]2013-05-23 16:47:52.032 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:47:52.234 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e01e0 [ CSSD]2013-05-23 16:47:52.234 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e01e0) con(0x22f1d20) skgpid ospid 30005 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:47:52.794 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:53.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:53.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:54.818 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:55.142 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:47:55.142 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1859 (0x7f431bff9190) [ CSSD]2013-05-23 16:47:55.142 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9190 (RPC#1859) tag(743002c) sent to node 1 [ CSSD]2013-05-23 16:47:55.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:55.143 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1859 (0x7f431bff9190) [ CSSD]2013-05-23 16:47:55.143 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9190 (RPC#1859) state 6, flags 0x202 [ CSSD]2013-05-23 16:47:55.143 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9190, ret 0, client (nil) [ CSSD]2013-05-23 16:47:55.143 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1859 [ CSSD]2013-05-23 16:47:55.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:55.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:56.502 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:47:56.502 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:47:56.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:57.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:57.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:47:58.836 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:47:59.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:47:59.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:00.504 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:00.505 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:48:00.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:01.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:01.665 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:02.103 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:02.103 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014810, gin 1, member 0 [ CSSD]2013-05-23 16:48:02.103 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:48:02.103 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:48:02.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:02.110 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:48:02.110 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f19b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:48:02.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:03.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:03.670 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:04.900 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:05.166 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:48:05.166 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1860 (0x7f431bff9210) [ CSSD]2013-05-23 16:48:05.167 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9210 (RPC#1860) tag(744002c) sent to node 1 [ CSSD]2013-05-23 16:48:05.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:05.167 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1860 (0x7f431bff9210) [ CSSD]2013-05-23 16:48:05.167 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9210 (RPC#1860) state 6, flags 0x202 [ CSSD]2013-05-23 16:48:05.167 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9210, ret 0, client (nil) [ CSSD]2013-05-23 16:48:05.167 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1860 [ CSSD]2013-05-23 16:48:05.508 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:05.508 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:05.681 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:05.681 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:06.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:07.266 [1191196992] >TRACE: clssgmAllocProc: (0x22f1730) allocated [ CSSD]2013-05-23 16:48:07.267 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:48:07.267 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f1730 - 1,2,3 [ CSSD]2013-05-23 16:48:07.267 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x22f1730) pid(30163) version 11:1:1:4 [ CSSD]2013-05-23 16:48:07.273 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f1730), client(1/0x22dabd0) [ CSSD]2013-05-23 16:48:07.273 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabd0) [ CSSD]2013-05-23 16:48:07.273 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabd0 with con 0x22dada0, requested num -1 [ CSSD]2013-05-23 16:48:07.273 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:07.273 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1861 (0x7f431bff9290) [ CSSD]2013-05-23 16:48:07.273 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9290 (RPC#1861) tag(745002c) sent to node 1 [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0745002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014811, gin 1, member 0 [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9290 (RPC#1861) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9290, ret 0, client 0x22dabd0 member 0x22d7820 [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabd0 [ CSSD]2013-05-23 16:48:07.274 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1861 [ CSSD]2013-05-23 16:48:07.281 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabd0) [ CSSD]2013-05-23 16:48:07.281 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:07.281 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7820) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:07.281 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1862 (0x7f431bff9310) [ CSSD]2013-05-23 16:48:07.281 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9310 (RPC#1862) tag(746002c) sent to node 1 [ CSSD]2013-05-23 16:48:07.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0746002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:07.281 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:07.281 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9310 (RPC#1862) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:07.281 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9310, ret 0, client 0x22dabd0 member 0x22d7820 [ CSSD]2013-05-23 16:48:07.281 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1862 [ CSSD]2013-05-23 16:48:07.281 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7820) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:48:07.484 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f1730 [ CSSD]2013-05-23 16:48:07.484 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f1730) con(0x231d6c0) skgpid ospid 30163 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:48:07.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:07.686 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:08.907 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:09.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:09.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:10.511 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:10.511 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:10.932 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:11.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:11.745 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:11.804 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c108370), client(2/0x22f1a70) [ CSSD]2013-05-23 16:48:11.804 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f1a70) [ CSSD]2013-05-23 16:48:11.805 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:48:11.805 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 16:48:12.945 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:13.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:13.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:14.949 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:15.191 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:48:15.191 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1863 (0x7f431bff9390) [ CSSD]2013-05-23 16:48:15.192 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9390 (RPC#1863) tag(747002c) sent to node 1 [ CSSD]2013-05-23 16:48:15.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:15.192 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1863 (0x7f431bff9390) [ CSSD]2013-05-23 16:48:15.192 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9390 (RPC#1863) state 6, flags 0x202 [ CSSD]2013-05-23 16:48:15.192 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9390, ret 0, client (nil) [ CSSD]2013-05-23 16:48:15.192 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1863 [ CSSD]2013-05-23 16:48:15.514 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:15.514 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:15.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:15.783 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:16.955 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:17.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:17.362 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014812, gin 1, member 0 [ CSSD]2013-05-23 16:48:17.362 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:48:17.362 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:48:17.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:17.369 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:48:17.369 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:48:17.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:17.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:19.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:19.516 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:19.516 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:48:19.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:19.841 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:21.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:21.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:21.846 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:22.516 [1191196992] >TRACE: clssgmAllocProc: (0x22f7b50) allocated [ CSSD]2013-05-23 16:48:22.517 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:48:22.517 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f7b50 - 1,2,3 [ CSSD]2013-05-23 16:48:22.517 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dada0) proc(0x22f7b50) pid(30309) version 11:1:1:4 [ CSSD]2013-05-23 16:48:22.523 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f7b50), client(1/0x22fc720) [ CSSD]2013-05-23 16:48:22.523 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fc720) [ CSSD]2013-05-23 16:48:22.523 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fc720 with con 0x22dabc0, requested num -1 [ CSSD]2013-05-23 16:48:22.523 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:22.523 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1864 (0x7f431bff9410) [ CSSD]2013-05-23 16:48:22.523 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9410 (RPC#1864) tag(748002c) sent to node 1 [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0748002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014813, gin 1, member 0 [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9410 (RPC#1864) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9410, ret 0, client 0x22fc720 member 0x22f8740 [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fc720 [ CSSD]2013-05-23 16:48:22.524 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1864 [ CSSD]2013-05-23 16:48:22.531 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fc720) [ CSSD]2013-05-23 16:48:22.531 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fc720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:22.531 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f8740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:22.531 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1865 (0x7f431bff9490) [ CSSD]2013-05-23 16:48:22.531 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9490 (RPC#1865) tag(749002c) sent to node 1 [ CSSD]2013-05-23 16:48:22.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0749002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:22.531 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:22.532 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9490 (RPC#1865) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:22.532 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9490, ret 0, client 0x22fc720 member 0x22f8740 [ CSSD]2013-05-23 16:48:22.532 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1865 [ CSSD]2013-05-23 16:48:22.532 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f8740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:48:22.572 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:22.572 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014814, gin 1, member 0 [ CSSD]2013-05-23 16:48:22.572 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:48:22.573 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:48:22.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:22.578 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:48:22.578 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22daa40) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:48:22.734 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f7b50 [ CSSD]2013-05-23 16:48:22.734 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f7b50) con(0x22dada0) skgpid ospid 30309 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:48:23.126 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:23.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:23.851 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:24.519 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:24.519 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:25.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:25.215 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:48:25.215 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1866 (0x7f431bff9510) [ CSSD]2013-05-23 16:48:25.215 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9510 (RPC#1866) tag(74a002c) sent to node 1 [ CSSD]2013-05-23 16:48:25.216 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:25.216 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1866 (0x7f431bff9510) [ CSSD]2013-05-23 16:48:25.216 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9510 (RPC#1866) state 6, flags 0x202 [ CSSD]2013-05-23 16:48:25.216 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9510, ret 0, client (nil) [ CSSD]2013-05-23 16:48:25.216 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1866 [ CSSD]2013-05-23 16:48:25.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:25.943 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:26.436 [1191196992] >TRACE: clssgmAllocProc: (0x22f7b50) allocated [ CSSD]2013-05-23 16:48:26.437 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:48:26.438 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f7b50 - 1,2,3 [ CSSD]2013-05-23 16:48:26.438 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dada0) proc(0x22f7b50) pid(30377) version 11:1:1:4 [ CSSD]2013-05-23 16:48:26.444 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f7b50), client(1/0x22fc720) [ CSSD]2013-05-23 16:48:26.444 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fc720) [ CSSD]2013-05-23 16:48:26.444 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fc720 with con 0x22dabc0, requested num -1 [ CSSD]2013-05-23 16:48:26.444 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:26.444 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1867 (0x7f431bff9590) [ CSSD]2013-05-23 16:48:26.444 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9590 (RPC#1867) tag(74b002c) sent to node 1 [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(074b002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014815, gin 1, member 0 [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9590 (RPC#1867) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9590, ret 0, client 0x22fc720 member 0x22f8740 [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fc720 [ CSSD]2013-05-23 16:48:26.445 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1867 [ CSSD]2013-05-23 16:48:26.452 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fc720) [ CSSD]2013-05-23 16:48:26.452 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fc720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:26.452 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f8740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:26.452 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1868 (0x7f431bff9610) [ CSSD]2013-05-23 16:48:26.452 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9610 (RPC#1868) tag(74c002c) sent to node 1 [ CSSD]2013-05-23 16:48:26.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(074c002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:26.452 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:26.453 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9610 (RPC#1868) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:26.453 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9610, ret 0, client 0x22fc720 member 0x22f8740 [ CSSD]2013-05-23 16:48:26.453 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1868 [ CSSD]2013-05-23 16:48:26.453 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f8740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:48:26.655 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f7b50 [ CSSD]2013-05-23 16:48:26.655 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f7b50) con(0x22dada0) skgpid ospid 30377 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:48:27.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:27.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:27.948 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:29.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:29.522 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:29.522 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:29.961 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:29.961 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:31.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:31.966 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:31.966 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:32.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:32.610 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014816, gin 1, member 0 [ CSSD]2013-05-23 16:48:32.610 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:48:32.610 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:48:32.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:32.616 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:48:32.616 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:48:33.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:33.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:33.971 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:34.525 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:34.525 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:35.242 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:48:35.242 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1869 (0x7f431bff9690) [ CSSD]2013-05-23 16:48:35.243 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9690 (RPC#1869) tag(74d002c) sent to node 1 [ CSSD]2013-05-23 16:48:35.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:35.243 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1869 (0x7f431bff9690) [ CSSD]2013-05-23 16:48:35.243 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9690 (RPC#1869) state 6, flags 0x202 [ CSSD]2013-05-23 16:48:35.243 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9690, ret 0, client (nil) [ CSSD]2013-05-23 16:48:35.243 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1869 [ CSSD]2013-05-23 16:48:35.253 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:35.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:35.982 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:37.258 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:37.767 [1191196992] >TRACE: clssgmAllocProc: (0x22f7b50) allocated [ CSSD]2013-05-23 16:48:37.768 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:48:37.769 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f7b50 - 1,2,3 [ CSSD]2013-05-23 16:48:37.769 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e01e0) proc(0x22f7b50) pid(30462) version 11:1:1:4 [ CSSD]2013-05-23 16:48:37.775 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f7b50), client(1/0x22f8670) [ CSSD]2013-05-23 16:48:37.775 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f8670) [ CSSD]2013-05-23 16:48:37.775 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f8670 with con 0x22dabc0, requested num -1 [ CSSD]2013-05-23 16:48:37.775 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:37.775 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1870 (0x7f431bff9710) [ CSSD]2013-05-23 16:48:37.775 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9710 (RPC#1870) tag(74e002c) sent to node 1 [ CSSD]2013-05-23 16:48:37.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(074e002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:37.775 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014817, gin 1, member 0 [ CSSD]2013-05-23 16:48:37.775 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:48:37.776 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:48:37.776 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:48:37.776 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9710 (RPC#1870) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:37.776 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9710, ret 0, client 0x22f8670 member 0x22daa40 [ CSSD]2013-05-23 16:48:37.776 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f8670 [ CSSD]2013-05-23 16:48:37.776 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1870 [ CSSD]2013-05-23 16:48:37.782 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f8670) [ CSSD]2013-05-23 16:48:37.782 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f8670), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:37.782 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22daa40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:37.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1871 (0x7f431bff9790) [ CSSD]2013-05-23 16:48:37.783 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9790 (RPC#1871) tag(74f002c) sent to node 1 [ CSSD]2013-05-23 16:48:37.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(074f002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:37.783 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:37.783 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9790 (RPC#1871) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:37.783 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9790, ret 0, client 0x22f8670 member 0x22daa40 [ CSSD]2013-05-23 16:48:37.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1871 [ CSSD]2013-05-23 16:48:37.783 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22daa40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:48:37.986 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f7b50 [ CSSD]2013-05-23 16:48:37.986 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f7b50) con(0x22e01e0) skgpid ospid 30462 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:48:37.987 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:37.987 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:39.309 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:39.528 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:39.528 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:40.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:40.050 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014818, gin 1, member 0 [ CSSD]2013-05-23 16:48:40.050 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:48:40.050 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:48:40.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:40.052 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:40.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:40.056 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:48:40.056 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22d7cd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:48:40.180 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:48:40.180 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:48:40.182 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:48:40.182 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:48:40.183 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:48:40.184 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:48:41.315 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:42.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:42.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:43.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:43.531 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:43.531 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:48:44.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:44.063 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:45.267 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:48:45.267 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1872 (0x7f431bff9810) [ CSSD]2013-05-23 16:48:45.268 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9810 (RPC#1872) tag(750002c) sent to node 1 [ CSSD]2013-05-23 16:48:45.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:45.268 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1872 (0x7f431bff9810) [ CSSD]2013-05-23 16:48:45.268 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9810 (RPC#1872) state 6, flags 0x202 [ CSSD]2013-05-23 16:48:45.268 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9810, ret 0, client (nil) [ CSSD]2013-05-23 16:48:45.268 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1872 [ CSSD]2013-05-23 16:48:45.333 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:46.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:46.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:47.338 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:47.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:47.859 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014819, gin 1, member 0 [ CSSD]2013-05-23 16:48:47.859 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:48:47.859 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:48:47.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:47.865 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:48:47.865 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22dacd0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:48:48.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:48.081 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:48.534 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:48.534 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:49.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:50.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:50.092 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:51.355 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:52.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:52.117 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:53.028 [1191196992] >TRACE: clssgmAllocProc: (0x22f7b50) allocated [ CSSD]2013-05-23 16:48:53.029 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:48:53.029 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f7b50 - 1,2,3 [ CSSD]2013-05-23 16:48:53.029 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7da0) proc(0x22f7b50) pid(30637) version 11:1:1:4 [ CSSD]2013-05-23 16:48:53.035 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22f7b50), client(1/0x22fc720) [ CSSD]2013-05-23 16:48:53.035 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fc720) [ CSSD]2013-05-23 16:48:53.035 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fc720 with con 0x22dabc0, requested num -1 [ CSSD]2013-05-23 16:48:53.035 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:53.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1873 (0x7f431bff9890) [ CSSD]2013-05-23 16:48:53.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9890 (RPC#1873) tag(751002c) sent to node 1 [ CSSD]2013-05-23 16:48:53.035 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0751002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:53.035 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014820, gin 1, member 0 [ CSSD]2013-05-23 16:48:53.035 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:48:53.035 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:48:53.036 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:48:53.036 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9890 (RPC#1873) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:53.036 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9890, ret 0, client 0x22fc720 member 0x22f8740 [ CSSD]2013-05-23 16:48:53.036 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fc720 [ CSSD]2013-05-23 16:48:53.036 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1873 [ CSSD]2013-05-23 16:48:53.042 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fc720) [ CSSD]2013-05-23 16:48:53.042 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fc720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:53.042 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f8740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:48:53.042 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1874 (0x7f431bff9910) [ CSSD]2013-05-23 16:48:53.042 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9910 (RPC#1874) tag(752002c) sent to node 1 [ CSSD]2013-05-23 16:48:53.042 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0752002c) incarnation(174631051) [ CSSD]2013-05-23 16:48:53.042 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:48:53.042 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9910 (RPC#1874) state 6, flags 0x100 [ CSSD]2013-05-23 16:48:53.042 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9910, ret 0, client 0x22fc720 member 0x22f8740 [ CSSD]2013-05-23 16:48:53.042 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1874 [ CSSD]2013-05-23 16:48:53.042 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f8740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:48:53.245 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f7b50 [ CSSD]2013-05-23 16:48:53.245 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f7b50) con(0x22d7da0) skgpid ospid 30637 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:48:53.407 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:53.537 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:53.538 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:54.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:54.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:54.195 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:48:54.195 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2312950), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:48:54.195 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2312950), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:48:54.195 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:48:54.196 [1191196992] >TRACE: clssgmDeadProc: proc 0x2303710 [ CSSD]2013-05-23 16:48:54.196 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2303710) con(0x22f2a10) skgpid 25335 ospid 25335 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:48:55.292 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:48:55.292 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1875 (0x7f431bff9990) [ CSSD]2013-05-23 16:48:55.293 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9990 (RPC#1875) tag(753002c) sent to node 1 [ CSSD]2013-05-23 16:48:55.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:55.293 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1875 (0x7f431bff9990) [ CSSD]2013-05-23 16:48:55.293 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9990 (RPC#1875) state 6, flags 0x202 [ CSSD]2013-05-23 16:48:55.293 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9990, ret 0, client (nil) [ CSSD]2013-05-23 16:48:55.293 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1875 [ CSSD]2013-05-23 16:48:55.433 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:56.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:56.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:57.438 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:48:58.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:48:58.162 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:48:58.540 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:48:58.540 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:48:59.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:00.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:00.173 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:01.454 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:02.179 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:02.179 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:03.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:03.120 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014821, gin 1, member 0 [ CSSD]2013-05-23 16:49:03.120 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:49:03.120 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:49:03.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:03.127 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:49:03.127 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:49:03.507 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:03.543 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:03.543 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:04.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:04.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:05.315 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:49:05.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1876 (0x7f431bff9a10) [ CSSD]2013-05-23 16:49:05.316 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9a10 (RPC#1876) tag(754002c) sent to node 1 [ CSSD]2013-05-23 16:49:05.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:05.316 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1876 (0x7f431bff9a10) [ CSSD]2013-05-23 16:49:05.316 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9a10 (RPC#1876) state 6, flags 0x202 [ CSSD]2013-05-23 16:49:05.316 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9a10, ret 0, client (nil) [ CSSD]2013-05-23 16:49:05.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1876 [ CSSD]2013-05-23 16:49:05.513 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:06.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:06.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:07.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:08.277 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 16:49:08.278 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:49:08.279 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 16:49:08.279 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(30799) version 11:1:1:4 [ CSSD]2013-05-23 16:49:08.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:08.282 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:08.285 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 16:49:08.285 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:49:08.285 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 16:49:08.285 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:08.285 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1877 (0x7f431bff9a90) [ CSSD]2013-05-23 16:49:08.285 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9a90 (RPC#1877) tag(755002c) sent to node 1 [ CSSD]2013-05-23 16:49:08.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0755002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:08.285 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014822, gin 1, member 0 [ CSSD]2013-05-23 16:49:08.285 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:49:08.286 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:49:08.286 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:49:08.286 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9a90 (RPC#1877) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:08.286 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9a90, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 16:49:08.286 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 16:49:08.286 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1877 [ CSSD]2013-05-23 16:49:08.292 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:49:08.292 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:08.292 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:08.292 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1878 (0x7f431bff9b10) [ CSSD]2013-05-23 16:49:08.292 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9b10 (RPC#1878) tag(756002c) sent to node 1 [ CSSD]2013-05-23 16:49:08.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0756002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:08.292 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:08.292 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9b10 (RPC#1878) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:08.292 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9b10, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 16:49:08.292 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1878 [ CSSD]2013-05-23 16:49:08.292 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:49:08.495 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 16:49:08.495 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 30799 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:49:08.546 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:08.546 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:09.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:10.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:10.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:11.536 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:12.319 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:12.319 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:13.548 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:13.549 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:13.550 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:14.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:14.351 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:15.338 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:49:15.338 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1879 (0x7f431bff9b90) [ CSSD]2013-05-23 16:49:15.339 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9b90 (RPC#1879) tag(757002c) sent to node 1 [ CSSD]2013-05-23 16:49:15.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:15.339 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1879 (0x7f431bff9b90) [ CSSD]2013-05-23 16:49:15.339 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9b90 (RPC#1879) state 6, flags 0x202 [ CSSD]2013-05-23 16:49:15.339 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9b90, ret 0, client (nil) [ CSSD]2013-05-23 16:49:15.339 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1879 [ CSSD]2013-05-23 16:49:15.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:16.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:16.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:17.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:18.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:18.371 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014823, gin 1, member 0 [ CSSD]2013-05-23 16:49:18.371 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:49:18.371 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:49:18.378 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:18.378 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:49:18.378 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:49:18.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:18.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:18.552 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:18.553 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:19.630 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:20.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:20.394 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:21.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:22.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:22.400 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:22.786 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 16:49:22.787 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:49:22.788 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 16:49:22.788 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c13ea10) pid(29991/29991) version 11:1:1:4 [ CSSD]2013-05-23 16:49:22.794 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ea10), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 16:49:22.794 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:49:22.794 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:49:22.794 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:49:22.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:22.844 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014824, gin 1, member 0 [ CSSD]2013-05-23 16:49:22.844 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:49:22.844 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:49:22.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:22.851 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:49:22.851 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:49:23.048 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ea10), client(2/0x7f431c0cf720) [ CSSD]2013-05-23 16:49:23.048 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 16:49:23.048 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:49:23.048 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 16:49:23.527 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:49:23.528 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:49:23.528 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:49:23.528 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c1348f0) pid(30951) version 11:1:1:4 [ CSSD]2013-05-23 16:49:23.535 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x2317510) [ CSSD]2013-05-23 16:49:23.535 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317510) [ CSSD]2013-05-23 16:49:23.535 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317510 with con 0x231d790, requested num -1 [ CSSD]2013-05-23 16:49:23.535 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:23.535 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1880 (0x7f431bff9c10) [ CSSD]2013-05-23 16:49:23.535 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9c10 (RPC#1880) tag(758002c) sent to node 1 [ CSSD]2013-05-23 16:49:23.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0758002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:23.535 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014825, gin 1, member 0 [ CSSD]2013-05-23 16:49:23.535 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:49:23.536 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:49:23.536 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:49:23.536 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9c10 (RPC#1880) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:23.536 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9c10, ret 0, client 0x2317510 member 0x22e2eb0 [ CSSD]2013-05-23 16:49:23.536 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317510 [ CSSD]2013-05-23 16:49:23.536 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1880 [ CSSD]2013-05-23 16:49:23.542 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317510) [ CSSD]2013-05-23 16:49:23.542 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317510), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:23.542 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e2eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:23.542 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1881 (0x7f431bff9c90) [ CSSD]2013-05-23 16:49:23.542 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9c90 (RPC#1881) tag(759002c) sent to node 1 [ CSSD]2013-05-23 16:49:23.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0759002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:23.543 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:23.543 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9c90 (RPC#1881) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:23.543 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9c90, ret 0, client 0x2317510 member 0x22e2eb0 [ CSSD]2013-05-23 16:49:23.543 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1881 [ CSSD]2013-05-23 16:49:23.543 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e2eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:49:23.555 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:23.556 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:23.745 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:49:23.745 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0e5200) skgpid ospid 30951 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:49:23.749 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:24.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:24.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:25.404 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:49:25.405 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1882 (0x7f431bff9d10) [ CSSD]2013-05-23 16:49:25.405 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9d10 (RPC#1882) tag(75a002c) sent to node 1 [ CSSD]2013-05-23 16:49:25.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:25.405 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1882 (0x7f431bff9d10) [ CSSD]2013-05-23 16:49:25.405 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9d10 (RPC#1882) state 6, flags 0x202 [ CSSD]2013-05-23 16:49:25.405 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9d10, ret 0, client (nil) [ CSSD]2013-05-23 16:49:25.405 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1882 [ CSSD]2013-05-23 16:49:25.795 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:26.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:26.457 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:26.697 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 16:49:26.698 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:49:26.698 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 16:49:26.698 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22dacd0) pid(31013) version 11:1:1:4 [ CSSD]2013-05-23 16:49:26.704 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22dacd0), client(1/0x22d7da0) [ CSSD]2013-05-23 16:49:26.704 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 16:49:26.704 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7da0 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 16:49:26.704 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:26.704 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1883 (0x7f431bff9d90) [ CSSD]2013-05-23 16:49:26.704 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9d90 (RPC#1883) tag(75b002c) sent to node 1 [ CSSD]2013-05-23 16:49:26.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(075b002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:26.704 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014826, gin 1, member 0 [ CSSD]2013-05-23 16:49:26.704 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:49:26.705 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:49:26.705 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:49:26.705 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9d90 (RPC#1883) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:26.705 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9d90, ret 0, client 0x22d7da0 member 0x22d7750 [ CSSD]2013-05-23 16:49:26.705 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7da0 [ CSSD]2013-05-23 16:49:26.705 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1883 [ CSSD]2013-05-23 16:49:26.711 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 16:49:26.711 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7da0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:26.711 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:26.711 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1884 (0x7f431bff9e10) [ CSSD]2013-05-23 16:49:26.711 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9e10 (RPC#1884) tag(75c002c) sent to node 1 [ CSSD]2013-05-23 16:49:26.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(075c002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:26.711 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:26.711 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9e10 (RPC#1884) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:26.711 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9e10, ret 0, client 0x22d7da0 member 0x22d7750 [ CSSD]2013-05-23 16:49:26.712 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1884 [ CSSD]2013-05-23 16:49:26.712 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:49:26.914 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 16:49:26.914 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2317510) skgpid ospid 31013 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:49:26.965 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:49:26.966 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:49:26.966 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:49:26.967 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:49:27.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:27.041 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014827, gin 1, member 0 [ CSSD]2013-05-23 16:49:27.041 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:49:27.041 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:49:27.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:27.047 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:49:27.047 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:49:27.558 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:27.558 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:49:27.807 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:28.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:28.482 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:29.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:30.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:30.554 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:31.819 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:32.562 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:32.562 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:32.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:32.579 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:33.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:33.628 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014828, gin 1, member 0 [ CSSD]2013-05-23 16:49:33.628 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:49:33.628 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:49:33.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:33.635 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:49:33.635 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:49:33.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:34.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:34.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:35.430 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:49:35.431 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1885 (0x7f431bff9e90) [ CSSD]2013-05-23 16:49:35.431 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9e90 (RPC#1885) tag(75d002c) sent to node 1 [ CSSD]2013-05-23 16:49:35.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:35.431 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1885 (0x7f431bff9e90) [ CSSD]2013-05-23 16:49:35.431 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9e90 (RPC#1885) state 6, flags 0x202 [ CSSD]2013-05-23 16:49:35.431 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bff9e90, ret 0, client (nil) [ CSSD]2013-05-23 16:49:35.431 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1885 [ CSSD]2013-05-23 16:49:35.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:36.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:36.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:37.565 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:37.565 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:37.917 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:38.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:38.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:38.786 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:49:38.787 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:49:38.787 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:49:38.787 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22d7cd0) pid(31097) version 11:1:1:4 [ CSSD]2013-05-23 16:49:38.793 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d7cd0), client(1/0x22dada0) [ CSSD]2013-05-23 16:49:38.793 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 16:49:38.793 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dada0 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 16:49:38.793 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:38.793 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1886 (0x7f431bff9f10) [ CSSD]2013-05-23 16:49:38.793 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9f10 (RPC#1886) tag(75e002c) sent to node 1 [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(075e002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014829, gin 1, member 0 [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9f10 (RPC#1886) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bff9f10, ret 0, client 0x22dada0 member 0x22d7750 [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dada0 [ CSSD]2013-05-23 16:49:38.794 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1886 [ CSSD]2013-05-23 16:49:38.801 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 16:49:38.801 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dada0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:38.801 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:38.801 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1887 (0x7f431bff9f90) [ CSSD]2013-05-23 16:49:38.801 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bff9f90 (RPC#1887) tag(75f002c) sent to node 1 [ CSSD]2013-05-23 16:49:38.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(075f002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:38.802 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:38.802 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bff9f90 (RPC#1887) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:38.802 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bff9f90, ret 0, client 0x22dada0 member 0x22d7750 [ CSSD]2013-05-23 16:49:38.802 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1887 [ CSSD]2013-05-23 16:49:38.802 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:49:39.005 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:49:39.005 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2317510) skgpid ospid 31097 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:49:39.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:40.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:40.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:41.948 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:42.568 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:42.568 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:42.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:42.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:43.959 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:44.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:44.671 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:45.454 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:49:45.454 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1888 (0x7f431bffa010) [ CSSD]2013-05-23 16:49:45.454 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa010 (RPC#1888) tag(760002c) sent to node 1 [ CSSD]2013-05-23 16:49:45.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:45.455 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1888 (0x7f431bffa010) [ CSSD]2013-05-23 16:49:45.455 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa010 (RPC#1888) state 6, flags 0x202 [ CSSD]2013-05-23 16:49:45.455 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa010, ret 0, client (nil) [ CSSD]2013-05-23 16:49:45.455 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1888 [ CSSD]2013-05-23 16:49:45.965 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:46.570 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:46.571 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:49:46.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:46.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:48.017 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:48.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:48.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:48.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:48.880 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014830, gin 1, member 0 [ CSSD]2013-05-23 16:49:48.880 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:49:48.881 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:49:48.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:48.889 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:49:48.889 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:49:50.024 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:50.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:50.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:51.574 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:51.574 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:52.028 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:52.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:52.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:54.037 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 16:49:54.038 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:49:54.038 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 16:49:54.038 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22dacd0) pid(31243) version 11:1:1:4 [ CSSD]2013-05-23 16:49:54.040 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:54.045 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22dacd0), client(1/0x23037e0) [ CSSD]2013-05-23 16:49:54.045 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23037e0) [ CSSD]2013-05-23 16:49:54.045 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23037e0 with con 0x22e2f70, requested num -1 [ CSSD]2013-05-23 16:49:54.045 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:54.045 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1889 (0x7f431bffa090) [ CSSD]2013-05-23 16:49:54.045 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa090 (RPC#1889) tag(761002c) sent to node 1 [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0761002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014831, gin 1, member 0 [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa090 (RPC#1889) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffa090, ret 0, client 0x23037e0 member 0x22e01e0 [ CSSD]2013-05-23 16:49:54.045 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23037e0 [ CSSD]2013-05-23 16:49:54.046 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1889 [ CSSD]2013-05-23 16:49:54.052 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23037e0) [ CSSD]2013-05-23 16:49:54.052 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23037e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:54.052 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:49:54.052 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1890 (0x7f431bffa110) [ CSSD]2013-05-23 16:49:54.052 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa110 (RPC#1890) tag(762002c) sent to node 1 [ CSSD]2013-05-23 16:49:54.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0762002c) incarnation(174631051) [ CSSD]2013-05-23 16:49:54.053 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:49:54.053 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa110 (RPC#1890) state 6, flags 0x100 [ CSSD]2013-05-23 16:49:54.053 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffa110, ret 0, client 0x23037e0 member 0x22e01e0 [ CSSD]2013-05-23 16:49:54.053 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1890 [ CSSD]2013-05-23 16:49:54.053 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:49:54.256 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 16:49:54.256 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2317510) skgpid ospid 31243 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:49:54.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:54.793 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:55.478 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:49:55.479 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1891 (0x7f431bffa190) [ CSSD]2013-05-23 16:49:55.479 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa190 (RPC#1891) tag(763002c) sent to node 1 [ CSSD]2013-05-23 16:49:55.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:55.479 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1891 (0x7f431bffa190) [ CSSD]2013-05-23 16:49:55.479 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa190 (RPC#1891) state 6, flags 0x202 [ CSSD]2013-05-23 16:49:55.479 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa190, ret 0, client (nil) [ CSSD]2013-05-23 16:49:55.480 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1891 [ CSSD]2013-05-23 16:49:56.046 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:56.577 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:49:56.577 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:49:56.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:56.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:49:58.118 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:49:58.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:49:58.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:00.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:00.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:00.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:01.580 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:01.580 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:02.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:02.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:02.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:04.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:04.135 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014832, gin 1, member 0 [ CSSD]2013-05-23 16:50:04.135 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:50:04.135 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:50:04.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:04.142 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:50:04.142 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:50:04.221 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:04.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:04.863 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:05.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:50:05.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1892 (0x7f431bffa210) [ CSSD]2013-05-23 16:50:05.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa210 (RPC#1892) tag(764002c) sent to node 1 [ CSSD]2013-05-23 16:50:05.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:05.557 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1892 (0x7f431bffa210) [ CSSD]2013-05-23 16:50:05.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa210 (RPC#1892) state 6, flags 0x202 [ CSSD]2013-05-23 16:50:05.557 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa210, ret 0, client (nil) [ CSSD]2013-05-23 16:50:05.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1892 [ CSSD]2013-05-23 16:50:05.583 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:05.583 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:50:06.227 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:06.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:06.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:07.966 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:07.966 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:50:07.967 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:07.967 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c1348f0) pid(31590) version 11:1:1:4 [ CSSD]2013-05-23 16:50:08.239 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:08.568 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:08.568 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0e5200) skgpid ospid 31590 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:08.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:08.894 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:09.287 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:09.288 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:50:09.289 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:09.289 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c1348f0) pid(31691) version 11:1:1:4 [ CSSD]2013-05-23 16:50:09.295 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:09.295 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:09.295 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:50:09.295 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:09.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1893 (0x7f431bffa290) [ CSSD]2013-05-23 16:50:09.296 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa290 (RPC#1893) tag(765002c) sent to node 1 [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0765002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014833, gin 1, member 0 [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa290 (RPC#1893) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffa290, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:50:09.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1893 [ CSSD]2013-05-23 16:50:09.303 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:09.303 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:09.303 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:09.303 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1894 (0x7f431bffa310) [ CSSD]2013-05-23 16:50:09.303 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa310 (RPC#1894) tag(766002c) sent to node 1 [ CSSD]2013-05-23 16:50:09.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0766002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:09.304 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:09.304 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa310 (RPC#1894) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:09.304 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffa310, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:09.304 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1894 [ CSSD]2013-05-23 16:50:09.304 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:50:09.506 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:09.506 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0e5200) skgpid ospid 31691 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:10.244 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:10.586 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:10.586 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:10.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:10.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:12.250 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:12.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:12.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:14.261 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:14.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:14.964 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:15.581 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:50:15.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1895 (0x7f431bffa390) [ CSSD]2013-05-23 16:50:15.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa390 (RPC#1895) tag(767002c) sent to node 1 [ CSSD]2013-05-23 16:50:15.582 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:15.582 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1895 (0x7f431bffa390) [ CSSD]2013-05-23 16:50:15.582 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa390 (RPC#1895) state 6, flags 0x202 [ CSSD]2013-05-23 16:50:15.582 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa390, ret 0, client (nil) [ CSSD]2013-05-23 16:50:15.582 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1895 [ CSSD]2013-05-23 16:50:15.589 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:15.589 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:16.267 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:16.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:16.970 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:18.319 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:18.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:18.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:19.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:19.398 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014834, gin 1, member 0 [ CSSD]2013-05-23 16:50:19.398 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:50:19.398 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:50:19.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:19.405 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:50:19.405 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:50:20.325 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:20.592 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:20.592 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:21.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:21.005 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:22.351 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:23.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:23.011 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:23.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:23.121 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014835, gin 1, member 0 [ CSSD]2013-05-23 16:50:23.121 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:50:23.121 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:50:23.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:23.128 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:50:23.128 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:50:24.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:24.539 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:24.540 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:50:24.541 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:24.541 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1348f0) pid(31862) version 11:1:1:4 [ CSSD]2013-05-23 16:50:24.547 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:24.547 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:24.547 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:50:24.547 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:24.547 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1896 (0x7f431bffa410) [ CSSD]2013-05-23 16:50:24.548 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa410 (RPC#1896) tag(768002c) sent to node 1 [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0768002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014836, gin 1, member 0 [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa410 (RPC#1896) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffa410, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:24.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:50:24.549 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1896 [ CSSD]2013-05-23 16:50:24.557 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:24.557 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:24.557 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:24.557 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1897 (0x7f431bffa490) [ CSSD]2013-05-23 16:50:24.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa490 (RPC#1897) tag(769002c) sent to node 1 [ CSSD]2013-05-23 16:50:24.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0769002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:24.557 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:24.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa490 (RPC#1897) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:24.557 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffa490, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:24.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1897 [ CSSD]2013-05-23 16:50:24.557 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:50:24.595 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:24.595 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:50:24.760 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:24.760 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0e6720) skgpid ospid 31862 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:25.065 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:25.065 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:25.606 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:50:25.606 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1898 (0x7f431bffa510) [ CSSD]2013-05-23 16:50:25.606 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa510 (RPC#1898) tag(76a002c) sent to node 1 [ CSSD]2013-05-23 16:50:25.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:25.607 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1898 (0x7f431bffa510) [ CSSD]2013-05-23 16:50:25.607 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa510 (RPC#1898) state 6, flags 0x202 [ CSSD]2013-05-23 16:50:25.607 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa510, ret 0, client (nil) [ CSSD]2013-05-23 16:50:25.607 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1898 [ CSSD]2013-05-23 16:50:26.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:26.962 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:26.963 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:50:26.963 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:26.963 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1348f0) pid(31917) version 11:1:1:4 [ CSSD]2013-05-23 16:50:26.969 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:26.969 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:26.969 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:50:26.969 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:26.969 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1899 (0x7f431bffa590) [ CSSD]2013-05-23 16:50:26.969 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa590 (RPC#1899) tag(76b002c) sent to node 1 [ CSSD]2013-05-23 16:50:26.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(076b002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014837, gin 1, member 0 [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa590 (RPC#1899) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffa590, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:50:26.970 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1899 [ CSSD]2013-05-23 16:50:26.976 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:26.976 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:26.976 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:26.976 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1900 (0x7f431bffa610) [ CSSD]2013-05-23 16:50:26.976 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa610 (RPC#1900) tag(76c002c) sent to node 1 [ CSSD]2013-05-23 16:50:26.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(076c002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:26.977 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:26.977 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa610 (RPC#1900) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:26.977 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffa610, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:26.977 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1900 [ CSSD]2013-05-23 16:50:26.977 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:50:27.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:27.091 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:27.179 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:27.179 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c108e60) skgpid ospid 31917 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:28.419 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:29.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:29.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:29.598 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:29.598 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:30.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:31.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:31.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:32.451 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:33.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:33.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:34.462 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:34.600 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:34.600 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:34.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:34.654 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014838, gin 1, member 0 [ CSSD]2013-05-23 16:50:34.654 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:50:34.654 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:50:34.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:34.661 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:50:34.661 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:50:35.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:35.184 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:35.629 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:50:35.629 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1901 (0x7f431bffa690) [ CSSD]2013-05-23 16:50:35.629 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa690 (RPC#1901) tag(76d002c) sent to node 1 [ CSSD]2013-05-23 16:50:35.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:35.630 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1901 (0x7f431bffa690) [ CSSD]2013-05-23 16:50:35.630 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa690 (RPC#1901) state 6, flags 0x202 [ CSSD]2013-05-23 16:50:35.630 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa690, ret 0, client (nil) [ CSSD]2013-05-23 16:50:35.630 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1901 [ CSSD]2013-05-23 16:50:36.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:37.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:37.190 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:38.520 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:39.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:39.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:39.603 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:39.603 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:39.791 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:39.792 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:50:39.792 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:39.792 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1348f0) pid(32027) version 11:1:1:4 [ CSSD]2013-05-23 16:50:39.798 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:39.798 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:39.798 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:50:39.798 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:39.798 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1902 (0x7f431bffa710) [ CSSD]2013-05-23 16:50:39.798 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa710 (RPC#1902) tag(76e002c) sent to node 1 [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(076e002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014839, gin 1, member 0 [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa710 (RPC#1902) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffa710, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:50:39.799 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1902 [ CSSD]2013-05-23 16:50:39.806 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:39.806 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:39.806 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:39.806 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1903 (0x7f431bffa790) [ CSSD]2013-05-23 16:50:39.806 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa790 (RPC#1903) tag(76f002c) sent to node 1 [ CSSD]2013-05-23 16:50:39.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(076f002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:39.807 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:39.807 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa790 (RPC#1903) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:39.807 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffa790, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:39.807 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1903 [ CSSD]2013-05-23 16:50:39.807 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:50:40.009 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:40.009 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c108e60) skgpid ospid 32027 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:40.525 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:41.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:41.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:42.549 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:43.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:43.232 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:44.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:44.606 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:44.606 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:45.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:45.266 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:45.653 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:50:45.654 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1904 (0x7f431bffa810) [ CSSD]2013-05-23 16:50:45.654 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa810 (RPC#1904) tag(770002c) sent to node 1 [ CSSD]2013-05-23 16:50:45.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:45.654 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1904 (0x7f431bffa810) [ CSSD]2013-05-23 16:50:45.654 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa810 (RPC#1904) state 6, flags 0x202 [ CSSD]2013-05-23 16:50:45.654 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa810, ret 0, client (nil) [ CSSD]2013-05-23 16:50:45.654 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1904 [ CSSD]2013-05-23 16:50:46.568 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:47.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:47.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:48.569 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:49.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:49.282 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:49.608 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:49.608 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:49.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:49.909 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014840, gin 1, member 0 [ CSSD]2013-05-23 16:50:49.909 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:50:49.909 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:50:49.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:49.916 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:50:49.916 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:50:50.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:51.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:51.288 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:52.599 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:53.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:53.335 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:53.611 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:53.611 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:50:54.633 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:55.041 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:55.042 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:50:55.042 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:55.042 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1348f0) pid(32170) version 11:1:1:4 [ CSSD]2013-05-23 16:50:55.047 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:55.047 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:55.047 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:50:55.047 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:55.047 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1905 (0x7f431bffa890) [ CSSD]2013-05-23 16:50:55.047 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa890 (RPC#1905) tag(771002c) sent to node 1 [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0771002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014841, gin 1, member 0 [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa890 (RPC#1905) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffa890, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:50:55.048 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1905 [ CSSD]2013-05-23 16:50:55.054 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:55.054 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:55.054 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:55.054 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1906 (0x7f431bffa910) [ CSSD]2013-05-23 16:50:55.054 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa910 (RPC#1906) tag(772002c) sent to node 1 [ CSSD]2013-05-23 16:50:55.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0772002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:55.055 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:55.055 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa910 (RPC#1906) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:55.055 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffa910, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:55.055 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1906 [ CSSD]2013-05-23 16:50:55.055 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:50:55.257 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:55.257 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c108e60) skgpid ospid 32170 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:55.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:55.366 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:55.681 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:50:55.681 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1907 (0x7f431bffa990) [ CSSD]2013-05-23 16:50:55.681 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffa990 (RPC#1907) tag(773002c) sent to node 1 [ CSSD]2013-05-23 16:50:55.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:55.682 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1907 (0x7f431bffa990) [ CSSD]2013-05-23 16:50:55.682 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffa990 (RPC#1907) state 6, flags 0x202 [ CSSD]2013-05-23 16:50:55.682 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffa990, ret 0, client (nil) [ CSSD]2013-05-23 16:50:55.682 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1907 [ CSSD]2013-05-23 16:50:56.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:57.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:57.371 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:50:57.659 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:50:57.660 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:50:57.660 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:50:57.660 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1348f0) pid(32225) version 11:1:1:4 [ CSSD]2013-05-23 16:50:57.677 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:57.677 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:57.677 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:50:57.677 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:57.677 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1908 (0x7f431bffaa10) [ CSSD]2013-05-23 16:50:57.677 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffaa10 (RPC#1908) tag(774002c) sent to node 1 [ CSSD]2013-05-23 16:50:57.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0774002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:57.677 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014842, gin 1, member 0 [ CSSD]2013-05-23 16:50:57.677 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:50:57.678 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:50:57.678 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:50:57.678 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffaa10 (RPC#1908) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:57.678 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffaa10, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:57.678 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:50:57.678 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1908 [ CSSD]2013-05-23 16:50:57.684 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:50:57.684 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:57.684 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:50:57.684 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1909 (0x7f431bffaa90) [ CSSD]2013-05-23 16:50:57.685 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffaa90 (RPC#1909) tag(775002c) sent to node 1 [ CSSD]2013-05-23 16:50:57.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0775002c) incarnation(174631051) [ CSSD]2013-05-23 16:50:57.685 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:50:57.685 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffaa90 (RPC#1909) state 6, flags 0x100 [ CSSD]2013-05-23 16:50:57.685 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffaa90, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:50:57.685 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1909 [ CSSD]2013-05-23 16:50:57.685 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:50:57.888 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:50:57.888 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0e6720) skgpid ospid 32225 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:50:58.614 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:50:58.614 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:50:58.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:50:59.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:50:59.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:00.657 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:01.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:01.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:02.663 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:03.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:03.415 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:03.617 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:03.617 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:04.674 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:05.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:05.164 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014843, gin 1, member 0 [ CSSD]2013-05-23 16:51:05.164 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:51:05.164 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:51:05.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:05.171 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:51:05.171 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:51:05.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:05.468 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:05.708 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:51:05.708 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1910 (0x7f431bffab10) [ CSSD]2013-05-23 16:51:05.709 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffab10 (RPC#1910) tag(776002c) sent to node 1 [ CSSD]2013-05-23 16:51:05.709 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:05.709 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1910 (0x7f431bffab10) [ CSSD]2013-05-23 16:51:05.709 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffab10 (RPC#1910) state 6, flags 0x202 [ CSSD]2013-05-23 16:51:05.709 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffab10, ret 0, client (nil) [ CSSD]2013-05-23 16:51:05.709 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1910 [ CSSD]2013-05-23 16:51:06.680 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:07.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:07.474 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:08.621 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:08.621 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:08.731 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:09.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:09.504 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:10.299 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:51:10.300 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:51:10.301 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:51:10.301 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1348f0) pid(32320) version 11:1:1:4 [ CSSD]2013-05-23 16:51:10.306 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:10.306 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:10.306 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:51:10.306 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:10.306 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1911 (0x7f431bffab90) [ CSSD]2013-05-23 16:51:10.307 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffab90 (RPC#1911) tag(777002c) sent to node 1 [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0777002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014844, gin 1, member 0 [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffab90 (RPC#1911) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffab90, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:51:10.307 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:51:10.308 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1911 [ CSSD]2013-05-23 16:51:10.314 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:10.314 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:10.314 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:10.314 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1912 (0x7f431bffac10) [ CSSD]2013-05-23 16:51:10.314 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffac10 (RPC#1912) tag(778002c) sent to node 1 [ CSSD]2013-05-23 16:51:10.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0778002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:10.314 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:10.314 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffac10 (RPC#1912) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:10.314 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffac10, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:51:10.315 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1912 [ CSSD]2013-05-23 16:51:10.315 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:51:10.517 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:51:10.517 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c108e60) skgpid ospid 32320 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:10.736 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:11.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:11.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:12.623 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:12.623 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:51:12.749 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:13.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:13.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:14.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:15.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:15.567 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:15.737 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:51:15.737 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1913 (0x7f431bffac90) [ CSSD]2013-05-23 16:51:15.737 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffac90 (RPC#1913) tag(779002c) sent to node 1 [ CSSD]2013-05-23 16:51:15.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:15.738 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1913 (0x7f431bffac90) [ CSSD]2013-05-23 16:51:15.738 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffac90 (RPC#1913) state 6, flags 0x202 [ CSSD]2013-05-23 16:51:15.738 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffac90, ret 0, client (nil) [ CSSD]2013-05-23 16:51:15.738 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1913 [ CSSD]2013-05-23 16:51:16.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:17.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:17.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:17.626 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:17.627 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:18.832 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:19.119 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:51:19.120 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:51:19.120 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:51:19.120 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c1348f0) pid(32426) version 11:1:1:4 [ CSSD]2013-05-23 16:51:19.127 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1348f0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:19.127 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:19.127 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 16:51:19.127 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:51:19.127 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1914 (0x7f431bffad10) [ CSSD]2013-05-23 16:51:19.127 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffad10 (RPC#1914) tag(77a002c) sent to node 1 [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(077a002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014845, gin 1, member 0 [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0823b0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffad10 (RPC#1914) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffad10, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:51:19.128 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1914 [ CSSD]2013-05-23 16:51:19.134 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:19.134 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:51:19.134 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0826b0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:51:19.134 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1915 (0x7f431bffad90) [ CSSD]2013-05-23 16:51:19.135 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffad90 (RPC#1915) tag(77b002c) sent to node 1 [ CSSD]2013-05-23 16:51:19.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(077b002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:19.135 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:51:19.135 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffad90 (RPC#1915) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:19.135 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffad90, ret 0, client 0x7f431c0e6e30 member 0x7f431c0826b0 [ CSSD]2013-05-23 16:51:19.135 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1915 [ CSSD]2013-05-23 16:51:19.135 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c0826b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:51:19.244 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ab30) allocated [ CSSD]2013-05-23 16:51:19.245 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:51:19.246 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ab30 - 1,2,3 [ CSSD]2013-05-23 16:51:19.246 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c13ab30) pid(32431) version 11:1:1:4 [ CSSD]2013-05-23 16:51:19.261 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:51:19.263 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:51:19.265 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ab30 [ CSSD]2013-05-23 16:51:19.265 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ab30) con(0x7f431c0e5200) skgpid ospid 32431 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:19.337 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:51:19.337 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c108e60) skgpid ospid 32426 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:19.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:19.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:20.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:20.419 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014846, gin 1, member 0 [ CSSD]2013-05-23 16:51:20.419 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:51:20.419 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:51:20.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:20.426 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:51:20.426 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:51:20.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:21.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:21.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:22.630 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:22.630 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:22.849 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:23.380 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:23.380 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014847, gin 1, member 0 [ CSSD]2013-05-23 16:51:23.380 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:51:23.380 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:51:23.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:23.388 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:51:23.388 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:51:23.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:23.617 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:24.262 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c129900) [ CSSD]2013-05-23 16:51:24.262 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c129900), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:51:24.262 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c129900), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:51:24.262 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 16:51:24.262 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:51:24.262 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c13f020) skgpid 27153 ospid 27153 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:24.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:25.549 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:51:25.551 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:51:25.551 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:51:25.551 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c129c20) pid(32481) version 11:1:1:4 [ CSSD]2013-05-23 16:51:25.567 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:25.567 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:25.567 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:51:25.567 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:25.567 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1916 (0x7f431bffae10) [ CSSD]2013-05-23 16:51:25.568 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffae10 (RPC#1916) tag(77c002c) sent to node 1 [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(077c002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014848, gin 1, member 0 [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffae10 (RPC#1916) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffae10, ret 0, client 0x7f431c0e6e30 member 0x7f431c095740 [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:51:25.568 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1916 [ CSSD]2013-05-23 16:51:25.575 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:25.575 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:25.575 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:25.575 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1917 (0x7f431bffae90) [ CSSD]2013-05-23 16:51:25.575 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffae90 (RPC#1917) tag(77d002c) sent to node 1 [ CSSD]2013-05-23 16:51:25.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(077d002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:25.576 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:25.576 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffae90 (RPC#1917) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:25.576 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffae90, ret 0, client 0x7f431c0e6e30 member 0x7f431c095740 [ CSSD]2013-05-23 16:51:25.576 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1917 [ CSSD]2013-05-23 16:51:25.576 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:51:25.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:25.617 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:25.761 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:51:25.762 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1918 (0x7f431bffaf10) [ CSSD]2013-05-23 16:51:25.762 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffaf10 (RPC#1918) tag(77e002c) sent to node 1 [ CSSD]2013-05-23 16:51:25.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:25.762 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1918 (0x7f431bffaf10) [ CSSD]2013-05-23 16:51:25.762 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffaf10 (RPC#1918) state 6, flags 0x202 [ CSSD]2013-05-23 16:51:25.762 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffaf10, ret 0, client (nil) [ CSSD]2013-05-23 16:51:25.762 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1918 [ CSSD]2013-05-23 16:51:25.778 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:51:25.778 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0e6720) skgpid ospid 32481 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:26.861 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:27.230 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:51:27.231 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:51:27.231 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:51:27.231 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c129c20) pid(32537) version 11:1:1:4 [ CSSD]2013-05-23 16:51:27.248 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:27.248 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:27.248 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:51:27.248 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:27.248 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1919 (0x7f431bffaf90) [ CSSD]2013-05-23 16:51:27.249 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffaf90 (RPC#1919) tag(77f002c) sent to node 1 [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(077f002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014849, gin 1, member 0 [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffaf90 (RPC#1919) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffaf90, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c300 [ CSSD]2013-05-23 16:51:27.249 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:51:27.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1919 [ CSSD]2013-05-23 16:51:27.257 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:51:27.257 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:27.257 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c300) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:27.257 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1920 (0x7f431bffb010) [ CSSD]2013-05-23 16:51:27.257 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb010 (RPC#1920) tag(780002c) sent to node 1 [ CSSD]2013-05-23 16:51:27.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0780002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:27.257 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:27.257 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb010 (RPC#1920) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:27.257 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb010, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c300 [ CSSD]2013-05-23 16:51:27.257 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1920 [ CSSD]2013-05-23 16:51:27.257 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c300) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:51:27.460 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:51:27.460 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0840c0) skgpid ospid 32537 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:27.633 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:27.634 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:27.644 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:27.644 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:28.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:29.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:29.676 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:30.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:31.636 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:31.636 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:51:31.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:31.682 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:32.930 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:33.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:33.708 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:34.935 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:35.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:35.673 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014850, gin 1, member 0 [ CSSD]2013-05-23 16:51:35.673 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:51:35.673 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:51:35.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:35.680 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:51:35.680 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:51:35.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:35.719 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:35.789 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:51:35.789 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1921 (0x7f431bffb090) [ CSSD]2013-05-23 16:51:35.789 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb090 (RPC#1921) tag(781002c) sent to node 1 [ CSSD]2013-05-23 16:51:35.790 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:35.790 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1921 (0x7f431bffb090) [ CSSD]2013-05-23 16:51:35.790 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb090 (RPC#1921) state 6, flags 0x202 [ CSSD]2013-05-23 16:51:35.790 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffb090, ret 0, client (nil) [ CSSD]2013-05-23 16:51:35.790 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1921 [ CSSD]2013-05-23 16:51:36.639 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:36.639 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:36.942 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:37.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:37.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:38.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:39.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:39.776 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:40.812 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:51:40.813 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:51:40.813 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:51:40.813 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22d7cd0) pid(32643) version 11:1:1:4 [ CSSD]2013-05-23 16:51:40.837 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7cd0), client(1/0x22dada0) [ CSSD]2013-05-23 16:51:40.837 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 16:51:40.837 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dada0 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 16:51:40.837 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:40.837 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1922 (0x7f431bffb110) [ CSSD]2013-05-23 16:51:40.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb110 (RPC#1922) tag(782002c) sent to node 1 [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0782002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014851, gin 1, member 0 [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb110 (RPC#1922) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb110, ret 0, client 0x22dada0 member 0x22d7750 [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dada0 [ CSSD]2013-05-23 16:51:40.838 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1922 [ CSSD]2013-05-23 16:51:40.844 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 16:51:40.844 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dada0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:40.844 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:40.845 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1923 (0x7f431bffb190) [ CSSD]2013-05-23 16:51:40.845 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb190 (RPC#1923) tag(783002c) sent to node 1 [ CSSD]2013-05-23 16:51:40.845 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0783002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:40.845 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:40.845 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb190 (RPC#1923) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:40.845 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb190, ret 0, client 0x22dada0 member 0x22d7750 [ CSSD]2013-05-23 16:51:40.845 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1923 [ CSSD]2013-05-23 16:51:40.845 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:51:40.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:41.047 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:51:41.047 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2317510) skgpid ospid 32643 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:41.642 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:41.642 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:41.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:41.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:43.030 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:43.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:43.808 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:45.035 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:45.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:45.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:45.820 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:51:45.820 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1924 (0x7f431bffb210) [ CSSD]2013-05-23 16:51:45.820 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb210 (RPC#1924) tag(784002c) sent to node 1 [ CSSD]2013-05-23 16:51:45.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:45.821 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1924 (0x7f431bffb210) [ CSSD]2013-05-23 16:51:45.821 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb210 (RPC#1924) state 6, flags 0x202 [ CSSD]2013-05-23 16:51:45.821 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffb210, ret 0, client (nil) [ CSSD]2013-05-23 16:51:45.821 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1924 [ CSSD]2013-05-23 16:51:46.645 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:46.645 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:47.060 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:47.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:47.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:49.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:49.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:49.897 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:50.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:50.927 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014852, gin 1, member 0 [ CSSD]2013-05-23 16:51:50.927 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:51:50.927 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:51:50.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:50.935 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:51:50.935 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:51:51.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:51.648 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:51.648 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:51:51.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:51.922 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:53.131 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:53.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:53.928 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:55.137 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:55.651 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:51:55.651 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:51:55.848 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:51:55.849 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1925 (0x7f431bffb290) [ CSSD]2013-05-23 16:51:55.849 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb290 (RPC#1925) tag(785002c) sent to node 1 [ CSSD]2013-05-23 16:51:55.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:55.849 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1925 (0x7f431bffb290) [ CSSD]2013-05-23 16:51:55.849 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb290 (RPC#1925) state 6, flags 0x202 [ CSSD]2013-05-23 16:51:55.849 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffb290, ret 0, client (nil) [ CSSD]2013-05-23 16:51:55.849 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1925 [ CSSD]2013-05-23 16:51:55.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:55.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:56.079 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 16:51:56.080 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:51:56.080 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 16:51:56.080 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22dacd0) pid(382) version 11:1:1:4 [ CSSD]2013-05-23 16:51:56.086 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22dacd0), client(1/0x22d7da0) [ CSSD]2013-05-23 16:51:56.086 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 16:51:56.086 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7da0 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 16:51:56.087 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:56.087 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1926 (0x7f431bffb310) [ CSSD]2013-05-23 16:51:56.087 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb310 (RPC#1926) tag(786002c) sent to node 1 [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0786002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014853, gin 1, member 0 [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb310 (RPC#1926) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb310, ret 0, client 0x22d7da0 member 0x22d7750 [ CSSD]2013-05-23 16:51:56.087 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7da0 [ CSSD]2013-05-23 16:51:56.088 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1926 [ CSSD]2013-05-23 16:51:56.095 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 16:51:56.095 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7da0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:56.095 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:51:56.095 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1927 (0x7f431bffb390) [ CSSD]2013-05-23 16:51:56.095 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb390 (RPC#1927) tag(787002c) sent to node 1 [ CSSD]2013-05-23 16:51:56.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0787002c) incarnation(174631051) [ CSSD]2013-05-23 16:51:56.095 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:51:56.095 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb390 (RPC#1927) state 6, flags 0x100 [ CSSD]2013-05-23 16:51:56.095 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb390, ret 0, client 0x22d7da0 member 0x22d7750 [ CSSD]2013-05-23 16:51:56.096 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1927 [ CSSD]2013-05-23 16:51:56.096 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:51:56.298 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 16:51:56.298 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2317510) skgpid ospid 382 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:51:57.142 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:51:58.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:51:58.004 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:51:59.154 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:00.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:00.096 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:00.654 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:00.654 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:01.160 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:02.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:02.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:03.181 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:04.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:04.108 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:05.187 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:05.657 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:05.658 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:05.872 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:52:05.873 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1928 (0x7f431bffb410) [ CSSD]2013-05-23 16:52:05.873 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb410 (RPC#1928) tag(788002c) sent to node 1 [ CSSD]2013-05-23 16:52:05.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:05.873 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1928 (0x7f431bffb410) [ CSSD]2013-05-23 16:52:05.873 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb410 (RPC#1928) state 6, flags 0x202 [ CSSD]2013-05-23 16:52:05.873 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffb410, ret 0, client (nil) [ CSSD]2013-05-23 16:52:05.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1928 [ CSSD]2013-05-23 16:52:06.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:06.119 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:06.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:06.184 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014854, gin 1, member 0 [ CSSD]2013-05-23 16:52:06.184 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:52:06.184 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:52:06.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:06.192 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:52:06.192 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:52:07.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:07.720 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:52:07.721 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:52:07.721 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:52:07.721 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22d7cd0) pid(512) version 11:1:1:4 [ CSSD]2013-05-23 16:52:07.727 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7cd0), client(1/0x2312950) [ CSSD]2013-05-23 16:52:07.727 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:52:07.727 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 16:52:07.727 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:07.727 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1929 (0x7f431bffb490) [ CSSD]2013-05-23 16:52:07.727 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb490 (RPC#1929) tag(789002c) sent to node 1 [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0789002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014855, gin 1, member 0 [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb490 (RPC#1929) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb490, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 16:52:07.728 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1929 [ CSSD]2013-05-23 16:52:07.735 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:52:07.735 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:07.735 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:07.735 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1930 (0x7f431bffb510) [ CSSD]2013-05-23 16:52:07.735 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb510 (RPC#1930) tag(78a002c) sent to node 1 [ CSSD]2013-05-23 16:52:07.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(078a002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:07.736 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:07.736 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb510 (RPC#1930) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:07.736 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb510, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:52:07.736 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1930 [ CSSD]2013-05-23 16:52:07.736 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:07.939 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:52:07.939 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x205ecb0) skgpid ospid 512 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:08.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:08.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:09.244 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:10.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:10.198 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:10.661 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:10.661 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:11.251 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:11.332 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 16:52:11.333 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:52:11.334 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 16:52:11.334 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(551) version 11:1:1:4 [ CSSD]2013-05-23 16:52:11.340 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 16:52:11.340 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:52:11.340 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 16:52:11.340 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:11.340 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1931 (0x7f431bffb590) [ CSSD]2013-05-23 16:52:11.340 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb590 (RPC#1931) tag(78b002c) sent to node 1 [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(078b002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014856, gin 1, member 0 [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb590 (RPC#1931) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb590, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 16:52:11.341 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1931 [ CSSD]2013-05-23 16:52:11.348 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:52:11.348 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:11.348 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:11.348 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1932 (0x7f431bffb610) [ CSSD]2013-05-23 16:52:11.348 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb610 (RPC#1932) tag(78c002c) sent to node 1 [ CSSD]2013-05-23 16:52:11.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(078c002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:11.349 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:11.349 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb610 (RPC#1932) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:11.349 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb610, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:52:11.349 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1932 [ CSSD]2013-05-23 16:52:11.349 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:11.551 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 16:52:11.551 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 551 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:12.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:12.204 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:13.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:14.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:14.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:14.663 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:14.664 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:52:15.267 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:15.896 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:52:15.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1933 (0x7f431bffb690) [ CSSD]2013-05-23 16:52:15.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb690 (RPC#1933) tag(78d002c) sent to node 1 [ CSSD]2013-05-23 16:52:15.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:15.897 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1933 (0x7f431bffb690) [ CSSD]2013-05-23 16:52:15.897 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb690 (RPC#1933) state 6, flags 0x202 [ CSSD]2013-05-23 16:52:15.897 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffb690, ret 0, client (nil) [ CSSD]2013-05-23 16:52:15.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1933 [ CSSD]2013-05-23 16:52:16.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:16.220 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:17.272 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:17.652 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:52:17.653 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:52:17.654 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:52:17.654 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c129c20) pid(847) version 11:1:1:4 [ CSSD]2013-05-23 16:52:17.801 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 16:52:17.802 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:52:17.803 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 16:52:17.803 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c1348f0) pid(857) version 11:1:1:4 [ CSSD]2013-05-23 16:52:17.819 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:52:17.821 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:52:17.824 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 16:52:17.824 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0840c0) skgpid ospid 857 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:17.898 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(73/0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:17.898 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 73 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:17.898 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c0e6e30 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:52:17.898 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:52:17.898 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1934 (0x7f431bffb710) [ CSSD]2013-05-23 16:52:17.898 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb710 (RPC#1934) tag(78e002c) sent to node 1 [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(078e002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014857, gin 1, member 0 [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c1215d0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb710 (RPC#1934) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb710, ret 0, client 0x7f431c0e6e30 member 0x7f431c095760 [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:52:17.899 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1934 [ CSSD]2013-05-23 16:52:17.905 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 73 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:17.905 [1191196992] >TRACE: clssgmExitGrock: client 73 (0x7f431c0e6e30), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:52:17.905 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095760) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:52:17.905 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1935 (0x7f431bffb790) [ CSSD]2013-05-23 16:52:17.905 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb790 (RPC#1935) tag(78f002c) sent to node 1 [ CSSD]2013-05-23 16:52:17.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(078f002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:17.905 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:52:17.905 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb790 (RPC#1935) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:17.905 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb790, ret 0, client 0x7f431c0e6e30 member 0x7f431c095760 [ CSSD]2013-05-23 16:52:17.906 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1935 [ CSSD]2013-05-23 16:52:17.906 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c095760) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:17.907 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:52:17.907 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c13f020) skgpid ospid 847 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:18.226 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:18.226 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:19.284 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:19.666 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:19.667 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:20.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:20.278 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:21.290 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:21.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:21.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014858, gin 1, member 0 [ CSSD]2013-05-23 16:52:21.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:52:21.442 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:52:21.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:21.450 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:52:21.450 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:52:22.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:22.303 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:23.342 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:23.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:23.637 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014859, gin 1, member 0 [ CSSD]2013-05-23 16:52:23.637 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:52:23.637 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:52:23.644 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:23.644 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:52:23.644 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:52:24.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:24.315 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:24.670 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:24.670 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:25.368 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:25.921 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:52:25.922 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1936 (0x7f431bffb810) [ CSSD]2013-05-23 16:52:25.922 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb810 (RPC#1936) tag(790002c) sent to node 1 [ CSSD]2013-05-23 16:52:25.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:25.922 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1936 (0x7f431bffb810) [ CSSD]2013-05-23 16:52:25.922 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb810 (RPC#1936) state 6, flags 0x202 [ CSSD]2013-05-23 16:52:25.922 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffb810, ret 0, client (nil) [ CSSD]2013-05-23 16:52:25.922 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1936 [ CSSD]2013-05-23 16:52:26.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:26.321 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:26.589 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:52:26.590 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:52:26.591 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:52:26.591 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c129c20) pid(927) version 11:1:1:4 [ CSSD]2013-05-23 16:52:26.597 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:26.597 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:26.597 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:52:26.597 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:26.597 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1937 (0x7f431bffb890) [ CSSD]2013-05-23 16:52:26.597 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb890 (RPC#1937) tag(791002c) sent to node 1 [ CSSD]2013-05-23 16:52:26.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0791002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014860, gin 1, member 0 [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb890 (RPC#1937) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb890, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c320 [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:52:26.598 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1937 [ CSSD]2013-05-23 16:52:26.605 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:26.605 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:26.605 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c320) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:26.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1938 (0x7f431bffb910) [ CSSD]2013-05-23 16:52:26.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb910 (RPC#1938) tag(792002c) sent to node 1 [ CSSD]2013-05-23 16:52:26.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0792002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:26.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:26.605 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb910 (RPC#1938) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:26.605 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffb910, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c320 [ CSSD]2013-05-23 16:52:26.605 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1938 [ CSSD]2013-05-23 16:52:26.605 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c320) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:26.808 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:52:26.808 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0840c0) skgpid ospid 927 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:27.374 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:27.499 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:52:27.501 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:52:27.501 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:52:27.501 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c129c20) pid(975) version 11:1:1:4 [ CSSD]2013-05-23 16:52:27.507 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:27.507 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:27.507 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:52:27.507 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:27.507 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1939 (0x7f431bffb990) [ CSSD]2013-05-23 16:52:27.507 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffb990 (RPC#1939) tag(793002c) sent to node 1 [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0793002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014861, gin 1, member 0 [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffb990 (RPC#1939) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffb990, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c320 [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:52:27.508 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1939 [ CSSD]2013-05-23 16:52:27.514 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:27.514 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:27.514 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c320) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:27.514 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1940 (0x7f431bffba10) [ CSSD]2013-05-23 16:52:27.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffba10 (RPC#1940) tag(794002c) sent to node 1 [ CSSD]2013-05-23 16:52:27.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0794002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:27.515 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:27.515 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffba10 (RPC#1940) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:27.515 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffba10, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c320 [ CSSD]2013-05-23 16:52:27.515 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1940 [ CSSD]2013-05-23 16:52:27.515 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c320) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:27.717 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:52:27.717 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0e6720) skgpid ospid 975 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:28.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:28.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:29.384 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:29.673 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:29.673 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:30.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:30.397 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:31.390 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:32.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:32.403 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:33.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:33.676 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:33.676 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:52:34.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:34.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:35.447 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:35.963 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:52:35.963 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1941 (0x7f431bffba90) [ CSSD]2013-05-23 16:52:35.963 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffba90 (RPC#1941) tag(795002c) sent to node 1 [ CSSD]2013-05-23 16:52:35.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:35.964 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1941 (0x7f431bffba90) [ CSSD]2013-05-23 16:52:35.964 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffba90 (RPC#1941) state 6, flags 0x202 [ CSSD]2013-05-23 16:52:35.964 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffba90, ret 0, client (nil) [ CSSD]2013-05-23 16:52:35.964 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1941 [ CSSD]2013-05-23 16:52:36.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:36.422 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:36.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:36.699 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014862, gin 1, member 0 [ CSSD]2013-05-23 16:52:36.699 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:52:36.699 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:52:36.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:36.707 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:52:36.707 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:52:37.492 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:38.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:38.448 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:38.679 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:38.679 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:39.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:40.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:40.448 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:41.571 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:41.839 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:52:41.840 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:52:41.841 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:52:41.841 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c129c20) pid(1072) version 11:1:1:4 [ CSSD]2013-05-23 16:52:41.847 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:41.847 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:41.847 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:52:41.847 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:41.847 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1942 (0x7f431bffbb10) [ CSSD]2013-05-23 16:52:41.847 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbb10 (RPC#1942) tag(796002c) sent to node 1 [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0796002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014863, gin 1, member 0 [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbb10 (RPC#1942) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:41.847 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffbb10, ret 0, client 0x7f431c0e6e30 member 0x7f431c095760 [ CSSD]2013-05-23 16:52:41.848 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:52:41.848 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1942 [ CSSD]2013-05-23 16:52:41.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:41.854 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:41.854 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095760) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:41.854 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1943 (0x7f431bffbb90) [ CSSD]2013-05-23 16:52:41.854 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbb90 (RPC#1943) tag(797002c) sent to node 1 [ CSSD]2013-05-23 16:52:41.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0797002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:41.854 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:41.854 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbb90 (RPC#1943) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:41.854 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffbb90, ret 0, client 0x7f431c0e6e30 member 0x7f431c095760 [ CSSD]2013-05-23 16:52:41.855 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1943 [ CSSD]2013-05-23 16:52:41.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095760) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:42.057 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:52:42.057 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0e6720) skgpid ospid 1072 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:42.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:42.473 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:43.582 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:43.681 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:43.682 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:44.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:44.506 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:45.588 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:45.987 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:52:45.987 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1944 (0x7f431bffbc10) [ CSSD]2013-05-23 16:52:45.987 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbc10 (RPC#1944) tag(798002c) sent to node 1 [ CSSD]2013-05-23 16:52:45.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:45.988 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1944 (0x7f431bffbc10) [ CSSD]2013-05-23 16:52:45.988 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbc10 (RPC#1944) state 6, flags 0x202 [ CSSD]2013-05-23 16:52:45.988 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffbc10, ret 0, client (nil) [ CSSD]2013-05-23 16:52:45.988 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1944 [ CSSD]2013-05-23 16:52:46.512 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:46.512 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:47.593 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:48.518 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:48.518 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:48.684 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:48.685 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:49.646 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:49.661 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c081f70) [ CSSD]2013-05-23 16:52:49.661 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c081f70), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:52:49.661 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c081f70), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:52:49.661 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 16:52:49.661 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137bb0 [ CSSD]2013-05-23 16:52:49.661 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137bb0) con(0x7f431c11ab50) skgpid 28124 ospid 28124 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:50.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:50.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:51.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:51.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:51.948 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014864, gin 1, member 0 [ CSSD]2013-05-23 16:52:51.948 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:52:51.948 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:52:51.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:51.956 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:52:51.956 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:52:52.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:52.536 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:53.663 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:53.688 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:53.688 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:54.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:54.587 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:55.668 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:56.013 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:52:56.013 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1945 (0x7f431bffbc90) [ CSSD]2013-05-23 16:52:56.014 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbc90 (RPC#1945) tag(799002c) sent to node 1 [ CSSD]2013-05-23 16:52:56.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:56.014 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1945 (0x7f431bffbc90) [ CSSD]2013-05-23 16:52:56.014 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbc90 (RPC#1945) state 6, flags 0x202 [ CSSD]2013-05-23 16:52:56.014 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffbc90, ret 0, client (nil) [ CSSD]2013-05-23 16:52:56.014 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1945 [ CSSD]2013-05-23 16:52:56.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:56.613 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:57.089 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:52:57.090 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:52:57.090 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:52:57.090 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c129c20) pid(1323) version 11:1:1:4 [ CSSD]2013-05-23 16:52:57.095 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:57.095 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:57.095 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 16:52:57.095 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:57.095 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1946 (0x7f431bffbd10) [ CSSD]2013-05-23 16:52:57.096 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbd10 (RPC#1946) tag(79a002c) sent to node 1 [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(079a002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014865, gin 1, member 0 [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbd10 (RPC#1946) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffbd10, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c230 [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:52:57.096 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1946 [ CSSD]2013-05-23 16:52:57.102 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:52:57.102 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:57.102 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:52:57.102 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1947 (0x7f431bffbd90) [ CSSD]2013-05-23 16:52:57.102 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbd90 (RPC#1947) tag(79b002c) sent to node 1 [ CSSD]2013-05-23 16:52:57.103 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(079b002c) incarnation(174631051) [ CSSD]2013-05-23 16:52:57.103 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:52:57.103 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbd90 (RPC#1947) state 6, flags 0x100 [ CSSD]2013-05-23 16:52:57.103 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffbd90, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c230 [ CSSD]2013-05-23 16:52:57.103 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1947 [ CSSD]2013-05-23 16:52:57.103 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:52:57.306 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:52:57.306 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c0e6720) skgpid ospid 1323 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:52:57.673 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:52:58.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:52:58.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:52:58.690 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:52:58.691 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:52:59.686 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:00.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:00.630 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:01.691 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:02.655 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:02.655 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:03.693 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:03.693 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:03.744 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:04.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:04.707 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:05.769 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:06.038 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:53:06.038 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1948 (0x7f431bffbe10) [ CSSD]2013-05-23 16:53:06.038 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbe10 (RPC#1948) tag(79c002c) sent to node 1 [ CSSD]2013-05-23 16:53:06.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:06.039 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1948 (0x7f431bffbe10) [ CSSD]2013-05-23 16:53:06.039 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbe10 (RPC#1948) state 6, flags 0x202 [ CSSD]2013-05-23 16:53:06.039 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffbe10, ret 0, client (nil) [ CSSD]2013-05-23 16:53:06.039 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1948 [ CSSD]2013-05-23 16:53:06.713 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:06.713 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:07.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:07.201 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014866, gin 1, member 0 [ CSSD]2013-05-23 16:53:07.201 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:53:07.201 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:53:07.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:07.209 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:53:07.209 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:53:07.780 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:08.696 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:08.697 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:08.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:08.719 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:09.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:10.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:10.731 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:11.792 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:12.348 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:53:12.348 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:53:12.349 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:53:12.349 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c129c20) pid(1484) version 11:1:1:4 [ CSSD]2013-05-23 16:53:12.365 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:53:12.365 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:53:12.365 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 16:53:12.365 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:12.365 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1949 (0x7f431bffbe90) [ CSSD]2013-05-23 16:53:12.366 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbe90 (RPC#1949) tag(79d002c) sent to node 1 [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(079d002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014867, gin 1, member 0 [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbe90 (RPC#1949) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffbe90, ret 0, client 0x7f431c0e6e30 member 0x7f431c095670 [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:53:12.366 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1949 [ CSSD]2013-05-23 16:53:12.373 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:53:12.373 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:12.373 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095670) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:12.373 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1950 (0x7f431bffbf10) [ CSSD]2013-05-23 16:53:12.373 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbf10 (RPC#1950) tag(79e002c) sent to node 1 [ CSSD]2013-05-23 16:53:12.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(079e002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:12.374 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:12.374 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbf10 (RPC#1950) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:12.374 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffbf10, ret 0, client 0x7f431c0e6e30 member 0x7f431c095670 [ CSSD]2013-05-23 16:53:12.374 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1950 [ CSSD]2013-05-23 16:53:12.374 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095670) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:53:12.576 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:53:12.576 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c13f020) skgpid ospid 1484 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:53:12.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:12.757 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:13.700 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:13.700 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:13.845 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:14.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:14.789 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:15.871 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:16.062 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:53:16.062 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1951 (0x7f431bffbf90) [ CSSD]2013-05-23 16:53:16.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffbf90 (RPC#1951) tag(79f002c) sent to node 1 [ CSSD]2013-05-23 16:53:16.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:16.063 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1951 (0x7f431bffbf90) [ CSSD]2013-05-23 16:53:16.063 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffbf90 (RPC#1951) state 6, flags 0x202 [ CSSD]2013-05-23 16:53:16.063 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffbf90, ret 0, client (nil) [ CSSD]2013-05-23 16:53:16.063 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1951 [ CSSD]2013-05-23 16:53:16.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:16.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:17.702 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:17.702 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:53:17.882 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:18.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:18.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:19.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:20.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:20.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:21.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:22.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:22.457 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014868, gin 1, member 0 [ CSSD]2013-05-23 16:53:22.457 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:53:22.457 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:53:22.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:22.465 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:53:22.465 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:53:22.705 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:22.705 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:22.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:22.837 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:23.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:23.898 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014869, gin 1, member 0 [ CSSD]2013-05-23 16:53:23.898 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:53:23.898 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:53:23.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:23.906 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:53:23.906 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:53:23.913 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:24.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:24.889 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:25.919 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:26.087 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:53:26.087 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1952 (0x7f431bffc010) [ CSSD]2013-05-23 16:53:26.087 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc010 (RPC#1952) tag(7a0002c) sent to node 1 [ CSSD]2013-05-23 16:53:26.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:26.088 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1952 (0x7f431bffc010) [ CSSD]2013-05-23 16:53:26.088 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc010 (RPC#1952) state 6, flags 0x202 [ CSSD]2013-05-23 16:53:26.088 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc010, ret 0, client (nil) [ CSSD]2013-05-23 16:53:26.088 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1952 [ CSSD]2013-05-23 16:53:26.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:26.895 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:27.608 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:53:27.609 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:53:27.610 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:53:27.610 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22d7cd0) pid(1626) version 11:1:1:4 [ CSSD]2013-05-23 16:53:27.625 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7cd0), client(1/0x22dada0) [ CSSD]2013-05-23 16:53:27.625 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 16:53:27.625 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dada0 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 16:53:27.626 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:27.626 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1953 (0x7f431bffc090) [ CSSD]2013-05-23 16:53:27.626 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc090 (RPC#1953) tag(7a1002c) sent to node 1 [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07a1002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014870, gin 1, member 0 [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc090 (RPC#1953) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc090, ret 0, client 0x22dada0 member 0x22d7750 [ CSSD]2013-05-23 16:53:27.626 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dada0 [ CSSD]2013-05-23 16:53:27.627 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1953 [ CSSD]2013-05-23 16:53:27.634 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 16:53:27.634 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dada0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:27.634 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:27.634 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1954 (0x7f431bffc110) [ CSSD]2013-05-23 16:53:27.634 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc110 (RPC#1954) tag(7a2002c) sent to node 1 [ CSSD]2013-05-23 16:53:27.634 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07a2002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:27.634 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:27.634 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc110 (RPC#1954) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:27.634 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffc110, ret 0, client 0x22dada0 member 0x22d7750 [ CSSD]2013-05-23 16:53:27.634 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1954 [ CSSD]2013-05-23 16:53:27.634 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:53:27.708 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:27.709 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:27.758 [1191196992] >TRACE: clssgmAllocProc: (0x22fc5a0) allocated [ CSSD]2013-05-23 16:53:27.759 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:53:27.759 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc5a0 - 1,2,3 [ CSSD]2013-05-23 16:53:27.759 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312940) proc(0x22fc5a0) pid(1673) version 11:1:1:4 [ CSSD]2013-05-23 16:53:27.765 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22fc5a0), client(1/0x22f8860) [ CSSD]2013-05-23 16:53:27.765 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f8860) [ CSSD]2013-05-23 16:53:27.765 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f8860 with con 0x22f7cd0, requested num -1 [ CSSD]2013-05-23 16:53:27.765 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:27.765 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1955 (0x7f431bffc190) [ CSSD]2013-05-23 16:53:27.766 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc190 (RPC#1955) tag(7a3002c) sent to node 1 [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07a3002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014871, gin 1, member 0 [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eec70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc190 (RPC#1955) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc190, ret 0, client 0x22f8860 member 0x22ff670 [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f8860 [ CSSD]2013-05-23 16:53:27.766 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1955 [ CSSD]2013-05-23 16:53:27.772 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f8860) [ CSSD]2013-05-23 16:53:27.772 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f8860), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:27.772 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ff670) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:27.772 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1956 (0x7f431bffc210) [ CSSD]2013-05-23 16:53:27.773 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc210 (RPC#1956) tag(7a4002c) sent to node 1 [ CSSD]2013-05-23 16:53:27.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07a4002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:27.773 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:27.773 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc210 (RPC#1956) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:27.773 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffc210, ret 0, client 0x22f8860 member 0x22ff670 [ CSSD]2013-05-23 16:53:27.773 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1956 [ CSSD]2013-05-23 16:53:27.773 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ff670) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:53:27.837 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:53:27.837 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2317510) skgpid ospid 1626 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:53:27.952 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:27.975 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc5a0 [ CSSD]2013-05-23 16:53:27.975 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc5a0) con(0x2312940) skgpid ospid 1673 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:53:28.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:28.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:29.958 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:30.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:30.932 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:31.963 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:32.711 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:32.711 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:32.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:32.938 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:33.974 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:34.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:34.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:35.981 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:36.111 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:53:36.111 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1957 (0x7f431bffc290) [ CSSD]2013-05-23 16:53:36.112 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc290 (RPC#1957) tag(7a5002c) sent to node 1 [ CSSD]2013-05-23 16:53:36.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:36.112 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1957 (0x7f431bffc290) [ CSSD]2013-05-23 16:53:36.112 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc290 (RPC#1957) state 6, flags 0x202 [ CSSD]2013-05-23 16:53:36.112 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc290, ret 0, client (nil) [ CSSD]2013-05-23 16:53:36.112 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1957 [ CSSD]2013-05-23 16:53:37.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:37.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:37.715 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:37.715 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:37.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:37.722 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014872, gin 1, member 0 [ CSSD]2013-05-23 16:53:37.722 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:53:37.722 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:53:37.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:37.728 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:53:37.729 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:53:37.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:39.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:39.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:39.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:40.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:40.361 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014873, gin 1, member 0 [ CSSD]2013-05-23 16:53:40.361 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:53:40.361 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:53:40.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:40.367 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:53:40.367 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:53:40.508 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:53:40.508 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:53:40.510 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:53:40.511 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:53:40.512 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:53:40.512 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:53:41.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:41.032 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:42.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:42.718 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:42.718 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:42.867 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 16:53:42.868 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:53:42.868 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 16:53:42.868 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22d7cd0) pid(1792) version 11:1:1:4 [ CSSD]2013-05-23 16:53:42.874 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7cd0), client(1/0x2312950) [ CSSD]2013-05-23 16:53:42.874 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:53:42.874 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 16:53:42.874 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:42.874 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1958 (0x7f431bffc310) [ CSSD]2013-05-23 16:53:42.874 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc310 (RPC#1958) tag(7a6002c) sent to node 1 [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07a6002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014874, gin 1, member 0 [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc310 (RPC#1958) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc310, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 16:53:42.875 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1958 [ CSSD]2013-05-23 16:53:42.881 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 16:53:42.881 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:42.881 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:42.881 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1959 (0x7f431bffc390) [ CSSD]2013-05-23 16:53:42.881 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc390 (RPC#1959) tag(7a7002c) sent to node 1 [ CSSD]2013-05-23 16:53:42.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07a7002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:42.881 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:42.882 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc390 (RPC#1959) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:42.882 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffc390, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 16:53:42.882 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1959 [ CSSD]2013-05-23 16:53:42.882 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:53:43.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:43.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:43.084 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 16:53:43.084 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x205ecb0) skgpid ospid 1792 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:53:44.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:45.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:45.089 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:46.101 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:46.134 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:53:46.135 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1960 (0x7f431bffc410) [ CSSD]2013-05-23 16:53:46.135 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc410 (RPC#1960) tag(7a8002c) sent to node 1 [ CSSD]2013-05-23 16:53:46.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:46.135 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1960 (0x7f431bffc410) [ CSSD]2013-05-23 16:53:46.135 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc410 (RPC#1960) state 6, flags 0x202 [ CSSD]2013-05-23 16:53:46.135 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc410, ret 0, client (nil) [ CSSD]2013-05-23 16:53:46.135 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1960 [ CSSD]2013-05-23 16:53:47.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:47.094 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:47.722 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:47.722 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:48.153 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:49.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:49.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:50.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:51.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:51.152 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:52.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:52.725 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:52.725 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:52.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:52.977 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014875, gin 1, member 0 [ CSSD]2013-05-23 16:53:52.977 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:53:52.977 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:53:52.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:52.984 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:53:52.984 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:53:53.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:53.158 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:54.177 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:54.285 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f1a70) [ CSSD]2013-05-23 16:53:54.285 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f1a70), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:53:54.285 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f1a70), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:53:54.285 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 16:53:54.295 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:53:54.295 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:53:54.295 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:53:54.295 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 27 [ CSSD]2013-05-23 16:53:54.295 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 16:53:54.295 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c109eb0) skgpid 28819 ospid 28819 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:53:55.095 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 16:53:55.096 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:53:55.097 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 16:53:55.097 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22dacd0) pid(1923/1923) version 11:1:1:4 [ CSSD]2013-05-23 16:53:55.113 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22dacd0), client(1/0x7f431c0e6720) [ CSSD]2013-05-23 16:53:55.113 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e6720) [ CSSD]2013-05-23 16:53:55.113 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:53:55.113 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 28 [ CSSD]2013-05-23 16:53:55.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:55.191 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:56.158 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:53:56.159 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1961 (0x7f431bffc490) [ CSSD]2013-05-23 16:53:56.159 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc490 (RPC#1961) tag(7a9002c) sent to node 1 [ CSSD]2013-05-23 16:53:56.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:56.159 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1961 (0x7f431bffc490) [ CSSD]2013-05-23 16:53:56.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc490 (RPC#1961) state 6, flags 0x202 [ CSSD]2013-05-23 16:53:56.159 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc490, ret 0, client (nil) [ CSSD]2013-05-23 16:53:56.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1961 [ CSSD]2013-05-23 16:53:56.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:57.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:57.196 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:53:57.728 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:53:57.728 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:53:58.116 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:53:58.117 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:53:58.117 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:53:58.117 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c129c20) pid(1957) version 11:1:1:4 [ CSSD]2013-05-23 16:53:58.133 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c129c20), client(1/0x7f431c135120) [ CSSD]2013-05-23 16:53:58.133 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:53:58.133 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:53:58.133 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:58.133 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1962 (0x7f431bffc510) [ CSSD]2013-05-23 16:53:58.133 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc510 (RPC#1962) tag(7aa002c) sent to node 1 [ CSSD]2013-05-23 16:53:58.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07aa002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:58.133 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014876, gin 1, member 0 [ CSSD]2013-05-23 16:53:58.133 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:53:58.134 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:53:58.134 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:53:58.134 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc510 (RPC#1962) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:58.134 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc510, ret 0, client 0x7f431c135120 member 0x7f431c1298e0 [ CSSD]2013-05-23 16:53:58.134 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 16:53:58.134 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1962 [ CSSD]2013-05-23 16:53:58.141 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 16:53:58.141 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:58.141 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1298e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:53:58.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1963 (0x7f431bffc590) [ CSSD]2013-05-23 16:53:58.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc590 (RPC#1963) tag(7ab002c) sent to node 1 [ CSSD]2013-05-23 16:53:58.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07ab002c) incarnation(174631051) [ CSSD]2013-05-23 16:53:58.142 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:53:58.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc590 (RPC#1963) state 6, flags 0x100 [ CSSD]2013-05-23 16:53:58.142 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffc590, ret 0, client 0x7f431c135120 member 0x7f431c1298e0 [ CSSD]2013-05-23 16:53:58.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1963 [ CSSD]2013-05-23 16:53:58.142 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1298e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:53:58.254 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:53:58.344 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:53:58.344 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c11ab50) skgpid ospid 1957 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:53:59.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:53:59.196 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:00.259 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:01.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:01.202 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:02.284 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:02.731 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:02.731 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:03.227 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:03.227 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:04.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:05.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:05.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:06.183 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:54:06.183 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1964 (0x7f431bffc610) [ CSSD]2013-05-23 16:54:06.184 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc610 (RPC#1964) tag(7ac002c) sent to node 1 [ CSSD]2013-05-23 16:54:06.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:06.184 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1964 (0x7f431bffc610) [ CSSD]2013-05-23 16:54:06.184 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc610 (RPC#1964) state 6, flags 0x202 [ CSSD]2013-05-23 16:54:06.184 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc610, ret 0, client (nil) [ CSSD]2013-05-23 16:54:06.184 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1964 [ CSSD]2013-05-23 16:54:06.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:07.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:07.245 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:07.734 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:07.734 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:08.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:08.248 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014877, gin 1, member 0 [ CSSD]2013-05-23 16:54:08.248 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:54:08.248 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:54:08.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:08.255 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:54:08.255 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:54:08.455 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:09.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:09.299 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:10.480 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:11.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:11.304 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:12.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:12.737 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:12.738 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:13.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:13.330 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:13.386 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:54:13.387 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:54:13.388 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:54:13.388 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c129c20) pid(2123) version 11:1:1:4 [ CSSD]2013-05-23 16:54:13.394 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c129c20), client(1/0x7f431c1299b0) [ CSSD]2013-05-23 16:54:13.394 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c1299b0) [ CSSD]2013-05-23 16:54:13.394 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c1299b0 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 16:54:13.394 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:13.394 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1965 (0x7f431bffc690) [ CSSD]2013-05-23 16:54:13.394 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc690 (RPC#1965) tag(7ad002c) sent to node 1 [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07ad002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014878, gin 1, member 0 [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc690 (RPC#1965) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc690, ret 0, client 0x7f431c1299b0 member 0x7f431c135050 [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c1299b0 [ CSSD]2013-05-23 16:54:13.395 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1965 [ CSSD]2013-05-23 16:54:13.401 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c1299b0) [ CSSD]2013-05-23 16:54:13.401 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c1299b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:13.401 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135050) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:13.401 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1966 (0x7f431bffc710) [ CSSD]2013-05-23 16:54:13.401 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc710 (RPC#1966) tag(7ae002c) sent to node 1 [ CSSD]2013-05-23 16:54:13.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07ae002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:13.402 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:13.402 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc710 (RPC#1966) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:13.402 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffc710, ret 0, client 0x7f431c1299b0 member 0x7f431c135050 [ CSSD]2013-05-23 16:54:13.402 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1966 [ CSSD]2013-05-23 16:54:13.402 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135050) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:54:13.605 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:54:13.605 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c11ab50) skgpid ospid 2123 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:54:14.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:15.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:15.422 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:16.208 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:54:16.209 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1967 (0x7f431bffc790) [ CSSD]2013-05-23 16:54:16.209 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc790 (RPC#1967) tag(7af002c) sent to node 1 [ CSSD]2013-05-23 16:54:16.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:16.209 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1967 (0x7f431bffc790) [ CSSD]2013-05-23 16:54:16.209 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc790 (RPC#1967) state 6, flags 0x202 [ CSSD]2013-05-23 16:54:16.209 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc790, ret 0, client (nil) [ CSSD]2013-05-23 16:54:16.209 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1967 [ CSSD]2013-05-23 16:54:16.504 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:17.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:17.428 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:17.741 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:17.741 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:18.556 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:19.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:19.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:20.581 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:21.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:21.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:22.587 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:22.744 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:22.744 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:23.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:23.469 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:23.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:23.510 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014879, gin 1, member 0 [ CSSD]2013-05-23 16:54:23.510 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:54:23.510 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:54:23.518 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:23.518 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:54:23.518 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:54:24.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:24.167 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014880, gin 1, member 0 [ CSSD]2013-05-23 16:54:24.167 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:54:24.167 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:54:24.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:24.177 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:54:24.177 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:54:24.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:25.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:25.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:26.236 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:54:26.236 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1968 (0x7f431bffc810) [ CSSD]2013-05-23 16:54:26.236 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc810 (RPC#1968) tag(7b0002c) sent to node 1 [ CSSD]2013-05-23 16:54:26.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:26.237 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1968 (0x7f431bffc810) [ CSSD]2013-05-23 16:54:26.237 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc810 (RPC#1968) state 6, flags 0x202 [ CSSD]2013-05-23 16:54:26.237 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffc810, ret 0, client (nil) [ CSSD]2013-05-23 16:54:26.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1968 [ CSSD]2013-05-23 16:54:26.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:27.283 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:54:27.283 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:54:27.285 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:54:27.285 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:54:27.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:27.447 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014881, gin 1, member 0 [ CSSD]2013-05-23 16:54:27.447 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c1215d0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:54:27.447 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:54:27.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:27.455 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:54:27.455 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:54:27.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:27.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:27.747 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:27.748 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:28.018 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129c20) allocated [ CSSD]2013-05-23 16:54:28.019 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:54:28.020 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129c20 - 1,2,3 [ CSSD]2013-05-23 16:54:28.020 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c129c20) pid(2281) version 11:1:1:4 [ CSSD]2013-05-23 16:54:28.026 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c129c20), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 16:54:28.026 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:54:28.026 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 16:54:28.026 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:28.026 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1969 (0x7f431bffc890) [ CSSD]2013-05-23 16:54:28.026 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc890 (RPC#1969) tag(7b1002c) sent to node 1 [ CSSD]2013-05-23 16:54:28.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07b1002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:28.026 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014882, gin 1, member 0 [ CSSD]2013-05-23 16:54:28.026 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:54:28.027 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:54:28.027 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:54:28.027 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc890 (RPC#1969) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:28.027 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc890, ret 0, client 0x7f431c0e6e30 member 0x7f431c135050 [ CSSD]2013-05-23 16:54:28.027 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 16:54:28.027 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1969 [ CSSD]2013-05-23 16:54:28.035 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 16:54:28.035 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:28.035 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135050) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:28.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1970 (0x7f431bffc910) [ CSSD]2013-05-23 16:54:28.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc910 (RPC#1970) tag(7b2002c) sent to node 1 [ CSSD]2013-05-23 16:54:28.035 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07b2002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:28.035 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:28.035 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc910 (RPC#1970) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:28.035 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffc910, ret 0, client 0x7f431c0e6e30 member 0x7f431c135050 [ CSSD]2013-05-23 16:54:28.036 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1970 [ CSSD]2013-05-23 16:54:28.036 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135050) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:54:28.238 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129c20 [ CSSD]2013-05-23 16:54:28.238 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129c20) con(0x7f431c11ab50) skgpid ospid 2281 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:54:28.640 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 16:54:28.641 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:54:28.642 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 16:54:28.642 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312940) proc(0x22d7750) pid(2309) version 11:1:1:4 [ CSSD]2013-05-23 16:54:28.649 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7750), client(1/0x22d7880) [ CSSD]2013-05-23 16:54:28.649 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:54:28.649 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7880 with con 0x22fca90, requested num -1 [ CSSD]2013-05-23 16:54:28.649 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:28.649 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1971 (0x7f431bffc990) [ CSSD]2013-05-23 16:54:28.649 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffc990 (RPC#1971) tag(7b3002c) sent to node 1 [ CSSD]2013-05-23 16:54:28.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07b3002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:28.649 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014883, gin 1, member 0 [ CSSD]2013-05-23 16:54:28.649 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eec70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:54:28.649 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:54:28.650 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:54:28.650 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffc990 (RPC#1971) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:28.650 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffc990, ret 0, client 0x22d7880 member 0x22fc5a0 [ CSSD]2013-05-23 16:54:28.650 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7880 [ CSSD]2013-05-23 16:54:28.650 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1971 [ CSSD]2013-05-23 16:54:28.657 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:54:28.657 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7880), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:28.657 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:28.657 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1972 (0x7f431bffca10) [ CSSD]2013-05-23 16:54:28.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffca10 (RPC#1972) tag(7b4002c) sent to node 1 [ CSSD]2013-05-23 16:54:28.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07b4002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:28.658 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:28.658 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffca10 (RPC#1972) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:28.658 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffca10, ret 0, client 0x22d7880 member 0x22fc5a0 [ CSSD]2013-05-23 16:54:28.658 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1972 [ CSSD]2013-05-23 16:54:28.658 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:54:28.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:28.860 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 16:54:28.860 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x2312940) skgpid ospid 2309 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:54:29.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:29.520 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:30.760 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:31.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:31.564 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:32.750 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:32.751 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:32.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:33.570 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:33.570 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:34.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:35.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:35.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:36.262 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:54:36.262 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1973 (0x7f431bffca90) [ CSSD]2013-05-23 16:54:36.263 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffca90 (RPC#1973) tag(7b5002c) sent to node 1 [ CSSD]2013-05-23 16:54:36.263 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:36.263 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1973 (0x7f431bffca90) [ CSSD]2013-05-23 16:54:36.263 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffca90 (RPC#1973) state 6, flags 0x202 [ CSSD]2013-05-23 16:54:36.263 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffca90, ret 0, client (nil) [ CSSD]2013-05-23 16:54:36.263 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1973 [ CSSD]2013-05-23 16:54:36.805 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:37.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:37.609 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:37.754 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:37.754 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:38.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:38.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014884, gin 1, member 0 [ CSSD]2013-05-23 16:54:38.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:54:38.770 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:54:38.777 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:38.777 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:54:38.777 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:54:38.856 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:39.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:39.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:40.863 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:41.520 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c133060) [ CSSD]2013-05-23 16:54:41.520 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c133060), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:54:41.520 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c133060), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:54:41.520 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 16:54:41.531 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c133260) [ CSSD]2013-05-23 16:54:41.531 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c133260), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:54:41.531 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c133260), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:54:41.531 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 27 [ CSSD]2013-05-23 16:54:41.531 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 16:54:41.531 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c120ad0) skgpid 28653 ospid 28653 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:54:41.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:41.645 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:42.757 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:42.757 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:42.874 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:43.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:43.652 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:43.902 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 16:54:43.903 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:54:43.904 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 16:54:43.904 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312940) proc(0x22d7750) pid(2445) version 11:1:1:4 [ CSSD]2013-05-23 16:54:43.910 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22d7750), client(1/0x22d7880) [ CSSD]2013-05-23 16:54:43.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:54:43.910 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7880 with con 0x22fca90, requested num -1 [ CSSD]2013-05-23 16:54:43.910 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:43.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1974 (0x7f431bffcb10) [ CSSD]2013-05-23 16:54:43.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcb10 (RPC#1974) tag(7b6002c) sent to node 1 [ CSSD]2013-05-23 16:54:43.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07b6002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:43.910 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014885, gin 1, member 0 [ CSSD]2013-05-23 16:54:43.910 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eec70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:54:43.911 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:54:43.911 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:54:43.911 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcb10 (RPC#1974) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:43.911 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffcb10, ret 0, client 0x22d7880 member 0x22fc5a0 [ CSSD]2013-05-23 16:54:43.911 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7880 [ CSSD]2013-05-23 16:54:43.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1974 [ CSSD]2013-05-23 16:54:43.918 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:54:43.918 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7880), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:43.918 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:43.918 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1975 (0x7f431bffcb90) [ CSSD]2013-05-23 16:54:43.918 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcb90 (RPC#1975) tag(7b7002c) sent to node 1 [ CSSD]2013-05-23 16:54:43.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07b7002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:43.918 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:43.918 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcb90 (RPC#1975) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:43.918 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffcb90, ret 0, client 0x22d7880 member 0x22fc5a0 [ CSSD]2013-05-23 16:54:43.919 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1975 [ CSSD]2013-05-23 16:54:43.919 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:54:44.121 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 16:54:44.121 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x2312940) skgpid ospid 2445 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:54:44.879 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:45.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:45.704 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:46.316 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:54:46.316 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1976 (0x7f431bffcc10) [ CSSD]2013-05-23 16:54:46.316 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcc10 (RPC#1976) tag(7b8002c) sent to node 1 [ CSSD]2013-05-23 16:54:46.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:46.317 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1976 (0x7f431bffcc10) [ CSSD]2013-05-23 16:54:46.317 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcc10 (RPC#1976) state 6, flags 0x202 [ CSSD]2013-05-23 16:54:46.317 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffcc10, ret 0, client (nil) [ CSSD]2013-05-23 16:54:46.317 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1976 [ CSSD]2013-05-23 16:54:46.904 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:47.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:47.729 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:47.760 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:47.760 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:48.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:49.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:49.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:50.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:51.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:51.827 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:52.763 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:52.764 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:52.964 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:53.833 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:53.833 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:54.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:54.024 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014886, gin 1, member 0 [ CSSD]2013-05-23 16:54:54.024 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:54:54.024 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:54:54.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:54.032 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:54:54.032 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:54:54.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:55.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:55.844 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:56.340 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:54:56.340 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1977 (0x7f431bffcc90) [ CSSD]2013-05-23 16:54:56.340 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcc90 (RPC#1977) tag(7b9002c) sent to node 1 [ CSSD]2013-05-23 16:54:56.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:56.341 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1977 (0x7f431bffcc90) [ CSSD]2013-05-23 16:54:56.341 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcc90 (RPC#1977) state 6, flags 0x202 [ CSSD]2013-05-23 16:54:56.341 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffcc90, ret 0, client (nil) [ CSSD]2013-05-23 16:54:56.341 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1977 [ CSSD]2013-05-23 16:54:56.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:57.766 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:54:57.766 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:54:57.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:57.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:54:58.987 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:54:59.154 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 16:54:59.155 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:54:59.156 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 16:54:59.156 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d790) proc(0x22d7750) pid(2593) version 11:1:1:4 [ CSSD]2013-05-23 16:54:59.172 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22d7750), client(1/0x22d7880) [ CSSD]2013-05-23 16:54:59.172 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:54:59.172 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7880 with con 0x22fc720, requested num -1 [ CSSD]2013-05-23 16:54:59.172 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:59.172 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1978 (0x7f431bffcd10) [ CSSD]2013-05-23 16:54:59.172 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcd10 (RPC#1978) tag(7ba002c) sent to node 1 [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07ba002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014887, gin 1, member 0 [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eec70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcd10 (RPC#1978) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffcd10, ret 0, client 0x22d7880 member 0x22dfd50 [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7880 [ CSSD]2013-05-23 16:54:59.173 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1978 [ CSSD]2013-05-23 16:54:59.180 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:54:59.180 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7880), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:59.180 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22dfd50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:54:59.180 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1979 (0x7f431bffcd90) [ CSSD]2013-05-23 16:54:59.181 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcd90 (RPC#1979) tag(7bb002c) sent to node 1 [ CSSD]2013-05-23 16:54:59.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07bb002c) incarnation(174631051) [ CSSD]2013-05-23 16:54:59.181 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:54:59.181 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcd90 (RPC#1979) state 6, flags 0x100 [ CSSD]2013-05-23 16:54:59.181 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffcd90, ret 0, client 0x22d7880 member 0x22dfd50 [ CSSD]2013-05-23 16:54:59.181 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1979 [ CSSD]2013-05-23 16:54:59.181 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22dfd50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:54:59.384 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 16:54:59.384 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x231d790) skgpid ospid 2593 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:54:59.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:54:59.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:00.993 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:01.769 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:01.769 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:55:01.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:01.927 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:03.005 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:03.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:03.932 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:05.011 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:05.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:05.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:06.375 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:55:06.375 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1980 (0x7f431bffce10) [ CSSD]2013-05-23 16:55:06.375 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffce10 (RPC#1980) tag(7bc002c) sent to node 1 [ CSSD]2013-05-23 16:55:06.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:06.376 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1980 (0x7f431bffce10) [ CSSD]2013-05-23 16:55:06.376 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffce10 (RPC#1980) state 6, flags 0x202 [ CSSD]2013-05-23 16:55:06.376 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffce10, ret 0, client (nil) [ CSSD]2013-05-23 16:55:06.376 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1980 [ CSSD]2013-05-23 16:55:06.772 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:06.772 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:07.017 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:07.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:07.968 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:08.461 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 16:55:08.462 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:55:08.462 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 16:55:08.462 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312940) proc(0x22d7750) pid(2725) version 11:1:1:4 [ CSSD]2013-05-23 16:55:09.068 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:09.220 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 16:55:09.220 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x2312940) skgpid ospid 2725 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:55:09.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:09.287 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014888, gin 1, member 0 [ CSSD]2013-05-23 16:55:09.287 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:55:09.288 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:55:09.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:09.295 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:55:09.295 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:55:10.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:10.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:11.094 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:11.775 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:11.775 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:12.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:12.008 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:13.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:14.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:14.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:14.425 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 16:55:14.426 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:55:14.427 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 16:55:14.427 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312940) proc(0x22d7750) pid(2861) version 11:1:1:4 [ CSSD]2013-05-23 16:55:14.432 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22d7750), client(1/0x22d7880) [ CSSD]2013-05-23 16:55:14.432 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:55:14.432 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7880 with con 0x22fca90, requested num -1 [ CSSD]2013-05-23 16:55:14.432 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:14.432 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1981 (0x7f431bffce90) [ CSSD]2013-05-23 16:55:14.432 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffce90 (RPC#1981) tag(7bd002c) sent to node 1 [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07bd002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014889, gin 1, member 0 [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eec70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffce90 (RPC#1981) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffce90, ret 0, client 0x22d7880 member 0x22fc5a0 [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7880 [ CSSD]2013-05-23 16:55:14.433 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1981 [ CSSD]2013-05-23 16:55:14.440 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:55:14.440 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7880), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:14.440 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:14.440 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1982 (0x7f431bffcf10) [ CSSD]2013-05-23 16:55:14.440 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcf10 (RPC#1982) tag(7be002c) sent to node 1 [ CSSD]2013-05-23 16:55:14.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07be002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:14.440 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:14.440 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcf10 (RPC#1982) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:14.440 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffcf10, ret 0, client 0x22d7880 member 0x22fc5a0 [ CSSD]2013-05-23 16:55:14.441 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1982 [ CSSD]2013-05-23 16:55:14.441 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:55:14.643 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 16:55:14.643 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x2312940) skgpid ospid 2861 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:55:15.111 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:16.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:16.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:16.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:55:16.403 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1983 (0x7f431bffcf90) [ CSSD]2013-05-23 16:55:16.403 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffcf90 (RPC#1983) tag(7bf002c) sent to node 1 [ CSSD]2013-05-23 16:55:16.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:16.404 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1983 (0x7f431bffcf90) [ CSSD]2013-05-23 16:55:16.404 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffcf90 (RPC#1983) state 6, flags 0x202 [ CSSD]2013-05-23 16:55:16.404 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffcf90, ret 0, client (nil) [ CSSD]2013-05-23 16:55:16.404 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1983 [ CSSD]2013-05-23 16:55:16.778 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:16.779 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:17.115 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:18.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:18.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:19.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:19.722 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 16:55:19.722 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0cf720), grp DBHWPROD, member 1 [ CSSD]2013-05-23 16:55:19.722 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0cf720), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 16:55:19.722 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 22 [ CSSD]2013-05-23 16:55:19.728 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 16:55:19.728 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 16:55:19.728 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 16:55:19.728 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 26 [ CSSD]2013-05-23 16:55:19.728 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 16:55:19.728 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c109f70) skgpid 29991 ospid 29991 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:55:20.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:20.072 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:20.146 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 16:55:20.147 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:55:20.147 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 16:55:20.147 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312940) proc(0x22d7750) pid(2954/2954) version 11:1:1:4 [ CSSD]2013-05-23 16:55:20.163 [1191196992] >TRACE: clssgmRegisterClient: proc(55/0x22d7750), client(1/0x22d7880) [ CSSD]2013-05-23 16:55:20.163 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 16:55:20.163 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:55:20.163 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 27 [ CSSD]2013-05-23 16:55:20.495 [1191196992] >TRACE: clssgmAllocProc: (0x2302d20) allocated [ CSSD]2013-05-23 16:55:20.496 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:55:20.497 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302d20 - 1,2,3 [ CSSD]2013-05-23 16:55:20.497 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f8800) proc(0x2302d20) pid(2960/2960) version 11:1:1:4 [ CSSD]2013-05-23 16:55:20.513 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x2302d20), client(1/0x22f1d30) [ CSSD]2013-05-23 16:55:20.513 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 16:55:20.513 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:55:20.513 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 28 [ CSSD]2013-05-23 16:55:20.781 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:20.781 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:55:21.211 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:22.078 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:22.078 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:22.986 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x2302d20), client(2/0x2302640) [ CSSD]2013-05-23 16:55:22.986 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2302640) [ CSSD]2013-05-23 16:55:22.986 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:55:22.986 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 23 [ CSSD]2013-05-23 16:55:23.266 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:23.475 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22dacd0), client(2/0x22d6ed0) [ CSSD]2013-05-23 16:55:23.475 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22d6ed0) [ CSSD]2013-05-23 16:55:23.475 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:55:23.475 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 16:55:24.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:24.085 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:24.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:24.443 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014890, gin 1, member 0 [ CSSD]2013-05-23 16:55:24.443 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:55:24.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:55:24.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:24.450 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:55:24.450 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:55:24.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:24.543 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014891, gin 1, member 0 [ CSSD]2013-05-23 16:55:24.543 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:55:24.543 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:55:24.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:24.550 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:55:24.550 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:55:25.273 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:25.783 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:25.784 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:26.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:26.096 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:26.428 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:55:26.428 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1984 (0x7f431bffd010) [ CSSD]2013-05-23 16:55:26.428 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd010 (RPC#1984) tag(7c0002c) sent to node 1 [ CSSD]2013-05-23 16:55:26.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:26.429 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1984 (0x7f431bffd010) [ CSSD]2013-05-23 16:55:26.429 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd010 (RPC#1984) state 6, flags 0x202 [ CSSD]2013-05-23 16:55:26.429 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd010, ret 0, client (nil) [ CSSD]2013-05-23 16:55:26.429 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1984 [ CSSD]2013-05-23 16:55:27.278 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:28.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:28.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:28.274 [1191196992] >TRACE: clssgmAllocProc: (0x22f4b20) allocated [ CSSD]2013-05-23 16:55:28.275 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:55:28.275 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4b20 - 1,2,3 [ CSSD]2013-05-23 16:55:28.275 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7040) proc(0x22f4b20) pid(3013) version 11:1:1:4 [ CSSD]2013-05-23 16:55:28.281 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4b20), client(1/0x22f5160) [ CSSD]2013-05-23 16:55:28.281 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f5160) [ CSSD]2013-05-23 16:55:28.281 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f5160 with con 0x22d7280, requested num -1 [ CSSD]2013-05-23 16:55:28.281 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:28.281 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1985 (0x7f431bffd090) [ CSSD]2013-05-23 16:55:28.281 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd090 (RPC#1985) tag(7c1002c) sent to node 1 [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07c1002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014892, gin 1, member 0 [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec570/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd090 (RPC#1985) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd090, ret 0, client 0x22f5160 member 0x22f4a20 [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f5160 [ CSSD]2013-05-23 16:55:28.282 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1985 [ CSSD]2013-05-23 16:55:28.288 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f5160) [ CSSD]2013-05-23 16:55:28.288 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f5160), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:28.288 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4a20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:28.288 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1986 (0x7f431bffd110) [ CSSD]2013-05-23 16:55:28.288 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd110 (RPC#1986) tag(7c2002c) sent to node 1 [ CSSD]2013-05-23 16:55:28.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07c2002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:28.289 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:28.289 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd110 (RPC#1986) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:28.289 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffd110, ret 0, client 0x22f5160 member 0x22f4a20 [ CSSD]2013-05-23 16:55:28.289 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1986 [ CSSD]2013-05-23 16:55:28.289 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4a20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:55:28.491 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4b20 [ CSSD]2013-05-23 16:55:28.491 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4b20) con(0x22d7040) skgpid ospid 3013 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:55:29.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:29.673 [1191196992] >TRACE: clssgmAllocProc: (0x22f2ab0) allocated [ CSSD]2013-05-23 16:55:29.675 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:55:29.675 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2ab0 - 1,2,3 [ CSSD]2013-05-23 16:55:29.675 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d6dd0) proc(0x22f2ab0) pid(3034) version 11:1:1:4 [ CSSD]2013-05-23 16:55:29.682 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f2ab0), client(1/0x22d7290) [ CSSD]2013-05-23 16:55:29.682 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7290) [ CSSD]2013-05-23 16:55:29.682 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7290 with con 0x22f3e70, requested num -1 [ CSSD]2013-05-23 16:55:29.682 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:29.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1987 (0x7f431bffd190) [ CSSD]2013-05-23 16:55:29.682 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd190 (RPC#1987) tag(7c3002c) sent to node 1 [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07c3002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014893, gin 1, member 0 [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd190 (RPC#1987) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd190, ret 0, client 0x22d7290 member 0x22f29b0 [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7290 [ CSSD]2013-05-23 16:55:29.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1987 [ CSSD]2013-05-23 16:55:29.689 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7290) [ CSSD]2013-05-23 16:55:29.689 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7290), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:29.689 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f29b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:29.689 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1988 (0x7f431bffd210) [ CSSD]2013-05-23 16:55:29.689 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd210 (RPC#1988) tag(7c4002c) sent to node 1 [ CSSD]2013-05-23 16:55:29.690 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07c4002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:29.690 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:29.690 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd210 (RPC#1988) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:29.690 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffd210, ret 0, client 0x22d7290 member 0x22f29b0 [ CSSD]2013-05-23 16:55:29.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1988 [ CSSD]2013-05-23 16:55:29.690 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f29b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:55:29.892 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2ab0 [ CSSD]2013-05-23 16:55:29.892 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2ab0) con(0x22d6dd0) skgpid ospid 3034 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:55:30.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:30.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:30.787 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:30.787 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:31.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:32.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:32.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:33.386 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:34.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:34.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:35.392 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:35.790 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:35.790 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:36.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:36.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:36.453 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:55:36.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1989 (0x7f431bffd290) [ CSSD]2013-05-23 16:55:36.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd290 (RPC#1989) tag(7c5002c) sent to node 1 [ CSSD]2013-05-23 16:55:36.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:36.454 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1989 (0x7f431bffd290) [ CSSD]2013-05-23 16:55:36.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd290 (RPC#1989) state 6, flags 0x202 [ CSSD]2013-05-23 16:55:36.454 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd290, ret 0, client (nil) [ CSSD]2013-05-23 16:55:36.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1989 [ CSSD]2013-05-23 16:55:37.398 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:37.457 [1191196992] >TRACE: clssgmRegisterClient: proc(55/0x22d7750), client(2/0x22f37a0) [ CSSD]2013-05-23 16:55:37.457 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22f37a0) [ CSSD]2013-05-23 16:55:37.457 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:55:37.457 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 16:55:38.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:38.173 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:39.410 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:39.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:39.799 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014894, gin 1, member 0 [ CSSD]2013-05-23 16:55:39.799 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:55:39.799 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:55:39.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:39.807 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:55:39.807 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:55:40.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:40.224 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:40.793 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:40.793 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:41.416 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:42.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:42.250 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:43.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:44.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:44.261 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:44.796 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:44.796 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:55:44.923 [1191196992] >TRACE: clssgmAllocProc: (0x22f4ea0) allocated [ CSSD]2013-05-23 16:55:44.925 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:55:44.925 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4ea0 - 1,2,3 [ CSSD]2013-05-23 16:55:44.925 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3dd0) proc(0x22f4ea0) pid(3178) version 11:1:1:4 [ CSSD]2013-05-23 16:55:44.931 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4ea0), client(1/0x22ec8e0) [ CSSD]2013-05-23 16:55:44.931 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:55:44.931 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ec8e0 with con 0x22f5130, requested num -1 [ CSSD]2013-05-23 16:55:44.931 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:44.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1990 (0x7f431bffd310) [ CSSD]2013-05-23 16:55:44.931 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd310 (RPC#1990) tag(7c6002c) sent to node 1 [ CSSD]2013-05-23 16:55:44.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07c6002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:44.931 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014895, gin 1, member 0 [ CSSD]2013-05-23 16:55:44.931 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eca50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:55:44.932 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:55:44.932 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:55:44.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd310 (RPC#1990) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:44.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd310, ret 0, client 0x22ec8e0 member 0x22f4da0 [ CSSD]2013-05-23 16:55:44.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ec8e0 [ CSSD]2013-05-23 16:55:44.932 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1990 [ CSSD]2013-05-23 16:55:44.938 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:55:44.939 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ec8e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:44.939 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:55:44.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1991 (0x7f431bffd390) [ CSSD]2013-05-23 16:55:44.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd390 (RPC#1991) tag(7c7002c) sent to node 1 [ CSSD]2013-05-23 16:55:44.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07c7002c) incarnation(174631051) [ CSSD]2013-05-23 16:55:44.939 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:55:44.939 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd390 (RPC#1991) state 6, flags 0x100 [ CSSD]2013-05-23 16:55:44.939 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffd390, ret 0, client 0x22ec8e0 member 0x22f4da0 [ CSSD]2013-05-23 16:55:44.939 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1991 [ CSSD]2013-05-23 16:55:44.939 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:55:45.141 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4ea0 [ CSSD]2013-05-23 16:55:45.141 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4ea0) con(0x22f3dd0) skgpid ospid 3178 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:55:45.493 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:46.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:46.268 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:46.478 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:55:46.478 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1992 (0x7f431bffd410) [ CSSD]2013-05-23 16:55:46.478 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd410 (RPC#1992) tag(7c8002c) sent to node 1 [ CSSD]2013-05-23 16:55:46.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:46.479 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1992 (0x7f431bffd410) [ CSSD]2013-05-23 16:55:46.479 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd410 (RPC#1992) state 6, flags 0x202 [ CSSD]2013-05-23 16:55:46.479 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd410, ret 0, client (nil) [ CSSD]2013-05-23 16:55:46.479 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1992 [ CSSD]2013-05-23 16:55:47.499 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:48.275 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:48.275 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:49.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:49.799 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:49.799 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:50.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:50.344 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:51.515 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:52.350 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:52.350 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:53.568 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:54.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:54.361 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:54.801 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:54.802 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:55:55.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:55.066 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014896, gin 1, member 0 [ CSSD]2013-05-23 16:55:55.066 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:55:55.066 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:55:55.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:55.073 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:55:55.073 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:55:55.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:56.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:56.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:56.502 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:55:56.503 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1993 (0x7f431bffd490) [ CSSD]2013-05-23 16:55:56.503 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd490 (RPC#1993) tag(7c9002c) sent to node 1 [ CSSD]2013-05-23 16:55:56.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:56.504 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1993 (0x7f431bffd490) [ CSSD]2013-05-23 16:55:56.504 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd490 (RPC#1993) state 6, flags 0x202 [ CSSD]2013-05-23 16:55:56.504 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd490, ret 0, client (nil) [ CSSD]2013-05-23 16:55:56.504 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1993 [ CSSD]2013-05-23 16:55:57.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:58.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:55:58.393 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:55:59.590 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:55:59.805 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:55:59.805 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:00.183 [1191196992] >TRACE: clssgmAllocProc: (0x22f4f20) allocated [ CSSD]2013-05-23 16:56:00.184 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:56:00.185 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4f20 - 1,2,3 [ CSSD]2013-05-23 16:56:00.185 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3ea0) proc(0x22f4f20) pid(3330) version 11:1:1:4 [ CSSD]2013-05-23 16:56:00.191 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4f20), client(1/0x22ec8e0) [ CSSD]2013-05-23 16:56:00.191 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:56:00.191 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ec8e0 with con 0x22f4d30, requested num -1 [ CSSD]2013-05-23 16:56:00.191 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:00.191 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1994 (0x7f431bffd510) [ CSSD]2013-05-23 16:56:00.191 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd510 (RPC#1994) tag(7ca002c) sent to node 1 [ CSSD]2013-05-23 16:56:00.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07ca002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014897, gin 1, member 0 [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eca50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd510 (RPC#1994) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd510, ret 0, client 0x22ec8e0 member 0x22ecd50 [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ec8e0 [ CSSD]2013-05-23 16:56:00.192 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1994 [ CSSD]2013-05-23 16:56:00.198 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:56:00.198 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ec8e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:00.198 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ecd50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:00.198 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1995 (0x7f431bffd590) [ CSSD]2013-05-23 16:56:00.198 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd590 (RPC#1995) tag(7cb002c) sent to node 1 [ CSSD]2013-05-23 16:56:00.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07cb002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:00.198 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:00.198 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd590 (RPC#1995) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:00.198 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffd590, ret 0, client 0x22ec8e0 member 0x22ecd50 [ CSSD]2013-05-23 16:56:00.199 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1995 [ CSSD]2013-05-23 16:56:00.199 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ecd50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:56:00.401 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4f20 [ CSSD]2013-05-23 16:56:00.401 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4f20) con(0x22f3ea0) skgpid ospid 3330 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:00.425 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:00.425 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:01.596 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:02.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:02.431 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:03.608 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:03.807 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:03.807 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:56:04.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:04.442 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:05.614 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:06.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:06.468 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:06.528 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:56:06.529 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1996 (0x7f431bffd610) [ CSSD]2013-05-23 16:56:06.529 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd610 (RPC#1996) tag(7cc002c) sent to node 1 [ CSSD]2013-05-23 16:56:06.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:06.529 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1996 (0x7f431bffd610) [ CSSD]2013-05-23 16:56:06.529 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd610 (RPC#1996) state 6, flags 0x202 [ CSSD]2013-05-23 16:56:06.529 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd610, ret 0, client (nil) [ CSSD]2013-05-23 16:56:06.530 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1996 [ CSSD]2013-05-23 16:56:07.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:08.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:08.475 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:08.810 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:08.810 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:09.681 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:10.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:10.318 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014898, gin 1, member 0 [ CSSD]2013-05-23 16:56:10.318 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:56:10.318 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:56:10.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:10.326 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:56:10.326 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:56:10.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:10.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:11.687 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:12.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:12.532 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:13.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:13.813 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:13.813 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:14.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:14.543 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:15.434 [1191196992] >TRACE: clssgmAllocProc: (0x22f4f20) allocated [ CSSD]2013-05-23 16:56:15.435 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:56:15.436 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4f20 - 1,2,3 [ CSSD]2013-05-23 16:56:15.436 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71e0) proc(0x22f4f20) pid(3490) version 11:1:1:4 [ CSSD]2013-05-23 16:56:15.442 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4f20), client(1/0x22d7290) [ CSSD]2013-05-23 16:56:15.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7290) [ CSSD]2013-05-23 16:56:15.442 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7290 with con 0x22f51f0, requested num -1 [ CSSD]2013-05-23 16:56:15.442 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:15.442 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1997 (0x7f431bffd690) [ CSSD]2013-05-23 16:56:15.442 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd690 (RPC#1997) tag(7cd002c) sent to node 1 [ CSSD]2013-05-23 16:56:15.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07cd002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:15.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014899, gin 1, member 0 [ CSSD]2013-05-23 16:56:15.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec9c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:56:15.443 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:56:15.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:56:15.443 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd690 (RPC#1997) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:15.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd690, ret 0, client 0x22d7290 member 0x22f4e20 [ CSSD]2013-05-23 16:56:15.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7290 [ CSSD]2013-05-23 16:56:15.443 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1997 [ CSSD]2013-05-23 16:56:15.449 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7290) [ CSSD]2013-05-23 16:56:15.450 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7290), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:15.450 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4e20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:15.450 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1998 (0x7f431bffd710) [ CSSD]2013-05-23 16:56:15.450 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd710 (RPC#1998) tag(7ce002c) sent to node 1 [ CSSD]2013-05-23 16:56:15.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07ce002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:15.450 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:15.450 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd710 (RPC#1998) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:15.450 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffd710, ret 0, client 0x22d7290 member 0x22f4e20 [ CSSD]2013-05-23 16:56:15.450 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1998 [ CSSD]2013-05-23 16:56:15.450 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4e20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:56:15.652 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4f20 [ CSSD]2013-05-23 16:56:15.652 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4f20) con(0x22d71e0) skgpid ospid 3490 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:15.702 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:16.553 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:56:16.554 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1999 (0x7f431bffd790) [ CSSD]2013-05-23 16:56:16.554 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd790 (RPC#1999) tag(7cf002c) sent to node 1 [ CSSD]2013-05-23 16:56:16.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:16.555 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1999 (0x7f431bffd790) [ CSSD]2013-05-23 16:56:16.555 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd790 (RPC#1999) state 6, flags 0x202 [ CSSD]2013-05-23 16:56:16.555 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd790, ret 0, client (nil) [ CSSD]2013-05-23 16:56:16.555 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1999 [ CSSD]2013-05-23 16:56:16.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:16.568 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:17.708 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:18.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:18.574 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:18.816 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:18.816 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:19.414 [1191196992] >TRACE: clssgmAllocProc: (0x22f4ea0) allocated [ CSSD]2013-05-23 16:56:19.415 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:56:19.416 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4ea0 - 1,2,3 [ CSSD]2013-05-23 16:56:19.416 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3df0) proc(0x22f4ea0) pid(3565) version 11:1:1:4 [ CSSD]2013-05-23 16:56:19.422 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4ea0), client(1/0x22ec8e0) [ CSSD]2013-05-23 16:56:19.422 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:56:19.422 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22ec8e0 with con 0x22f5150, requested num -1 [ CSSD]2013-05-23 16:56:19.422 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:56:19.422 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2000 (0x7f431bffd810) [ CSSD]2013-05-23 16:56:19.422 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd810 (RPC#2000) tag(7d0002c) sent to node 1 [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07d0002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014900, gin 1, member 0 [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eca50/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd810 (RPC#2000) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd810, ret 0, client 0x22ec8e0 member 0x22f4da0 [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ec8e0 [ CSSD]2013-05-23 16:56:19.423 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2000 [ CSSD]2013-05-23 16:56:19.430 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:56:19.430 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ec8e0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:56:19.430 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4da0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 16:56:19.430 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2001 (0x7f431bffd890) [ CSSD]2013-05-23 16:56:19.430 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd890 (RPC#2001) tag(7d1002c) sent to node 1 [ CSSD]2013-05-23 16:56:19.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07d1002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:19.431 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 16:56:19.431 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd890 (RPC#2001) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:19.431 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffd890, ret 0, client 0x22ec8e0 member 0x22f4da0 [ CSSD]2013-05-23 16:56:19.431 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2001 [ CSSD]2013-05-23 16:56:19.431 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22f4da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:56:19.541 [1191196992] >TRACE: clssgmAllocProc: (0x23183e0) allocated [ CSSD]2013-05-23 16:56:19.542 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:56:19.543 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23183e0 - 1,2,3 [ CSSD]2013-05-23 16:56:19.543 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec950) proc(0x23183e0) pid(3570) version 11:1:1:4 [ CSSD]2013-05-23 16:56:19.558 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:56:19.560 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:56:19.561 [1191196992] >TRACE: clssgmDeadProc: proc 0x23183e0 [ CSSD]2013-05-23 16:56:19.562 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23183e0) con(0x22ec950) skgpid ospid 3570 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:19.633 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4ea0 [ CSSD]2013-05-23 16:56:19.633 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4ea0) con(0x22f3df0) skgpid ospid 3565 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:19.721 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:20.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:20.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:21.747 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:22.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:22.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:23.779 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:23.819 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:23.819 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:24.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:24.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:24.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:24.705 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014901, gin 1, member 0 [ CSSD]2013-05-23 16:56:24.705 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:56:24.705 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:56:24.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:24.714 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:56:24.714 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:56:25.588 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:25.588 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014902, gin 1, member 0 [ CSSD]2013-05-23 16:56:25.588 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:56:25.588 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:56:25.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:25.595 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:56:25.595 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:56:25.804 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:26.606 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:56:26.607 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2002 (0x7f431bffd910) [ CSSD]2013-05-23 16:56:26.607 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd910 (RPC#2002) tag(7d2002c) sent to node 1 [ CSSD]2013-05-23 16:56:26.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:26.608 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2002 (0x7f431bffd910) [ CSSD]2013-05-23 16:56:26.608 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd910 (RPC#2002) state 6, flags 0x202 [ CSSD]2013-05-23 16:56:26.608 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffd910, ret 0, client (nil) [ CSSD]2013-05-23 16:56:26.608 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2002 [ CSSD]2013-05-23 16:56:26.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:26.670 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:27.816 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:27.822 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:27.822 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:56:28.536 [1191196992] >TRACE: clssgmAllocProc: (0x22f4f20) allocated [ CSSD]2013-05-23 16:56:28.537 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:56:28.538 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4f20 - 1,2,3 [ CSSD]2013-05-23 16:56:28.538 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3e80) proc(0x22f4f20) pid(3659) version 11:1:1:4 [ CSSD]2013-05-23 16:56:28.544 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4f20), client(1/0x22ec8e0) [ CSSD]2013-05-23 16:56:28.544 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:56:28.544 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ec8e0 with con 0x22f4d30, requested num -1 [ CSSD]2013-05-23 16:56:28.544 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:28.544 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2003 (0x7f431bffd990) [ CSSD]2013-05-23 16:56:28.544 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffd990 (RPC#2003) tag(7d3002c) sent to node 1 [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07d3002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014903, gin 1, member 0 [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eca50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffd990 (RPC#2003) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffd990, ret 0, client 0x22ec8e0 member 0x22ecd50 [ CSSD]2013-05-23 16:56:28.545 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ec8e0 [ CSSD]2013-05-23 16:56:28.546 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2003 [ CSSD]2013-05-23 16:56:28.553 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ec8e0) [ CSSD]2013-05-23 16:56:28.553 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ec8e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:28.553 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ecd50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:28.553 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2004 (0x7f431bffda10) [ CSSD]2013-05-23 16:56:28.553 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffda10 (RPC#2004) tag(7d4002c) sent to node 1 [ CSSD]2013-05-23 16:56:28.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07d4002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:28.554 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:28.554 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffda10 (RPC#2004) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:28.554 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffda10, ret 0, client 0x22ec8e0 member 0x22ecd50 [ CSSD]2013-05-23 16:56:28.554 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2004 [ CSSD]2013-05-23 16:56:28.554 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ecd50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:56:28.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:28.696 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:28.756 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4f20 [ CSSD]2013-05-23 16:56:28.756 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4f20) con(0x22f3e80) skgpid ospid 3659 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:29.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:30.688 [1191196992] >TRACE: clssgmAllocProc: (0x22f4ea0) allocated [ CSSD]2013-05-23 16:56:30.689 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:56:30.690 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4ea0 - 1,2,3 [ CSSD]2013-05-23 16:56:30.690 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3db0) proc(0x22f4ea0) pid(3680) version 11:1:1:4 [ CSSD]2013-05-23 16:56:30.707 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4ea0), client(1/0x22ec8f0) [ CSSD]2013-05-23 16:56:30.707 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ec8f0) [ CSSD]2013-05-23 16:56:30.707 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ec8f0 with con 0x22f5130, requested num -1 [ CSSD]2013-05-23 16:56:30.707 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:30.707 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2005 (0x7f431bffda90) [ CSSD]2013-05-23 16:56:30.707 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffda90 (RPC#2005) tag(7d5002c) sent to node 1 [ CSSD]2013-05-23 16:56:30.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07d5002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:30.707 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014904, gin 1, member 0 [ CSSD]2013-05-23 16:56:30.707 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22eca60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:56:30.708 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:56:30.708 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:56:30.708 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffda90 (RPC#2005) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:30.708 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffda90, ret 0, client 0x22ec8f0 member 0x22f4da0 [ CSSD]2013-05-23 16:56:30.708 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ec8f0 [ CSSD]2013-05-23 16:56:30.708 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2005 [ CSSD]2013-05-23 16:56:30.714 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ec8f0) [ CSSD]2013-05-23 16:56:30.714 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ec8f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:30.714 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:30.714 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2006 (0x7f431bffdb10) [ CSSD]2013-05-23 16:56:30.714 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdb10 (RPC#2006) tag(7d6002c) sent to node 1 [ CSSD]2013-05-23 16:56:30.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07d6002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:30.715 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:30.715 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdb10 (RPC#2006) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:30.715 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffdb10, ret 0, client 0x22ec8f0 member 0x22f4da0 [ CSSD]2013-05-23 16:56:30.715 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2006 [ CSSD]2013-05-23 16:56:30.715 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:56:30.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:30.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:30.918 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4ea0 [ CSSD]2013-05-23 16:56:30.918 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4ea0) con(0x22f3db0) skgpid ospid 3680 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:31.828 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:32.733 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:32.733 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:32.825 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:32.825 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:33.879 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:34.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:34.744 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:35.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:36.634 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:56:36.635 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2007 (0x7f431bffdb90) [ CSSD]2013-05-23 16:56:36.635 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdb90 (RPC#2007) tag(7d7002c) sent to node 1 [ CSSD]2013-05-23 16:56:36.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:36.635 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2007 (0x7f431bffdb90) [ CSSD]2013-05-23 16:56:36.635 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdb90 (RPC#2007) state 6, flags 0x202 [ CSSD]2013-05-23 16:56:36.635 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffdb90, ret 0, client (nil) [ CSSD]2013-05-23 16:56:36.635 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2007 [ CSSD]2013-05-23 16:56:36.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:36.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:37.828 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:37.828 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:37.897 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:38.448 [1191196992] >TRACE: clssgmAllocProc: (0x22f4f20) allocated [ CSSD]2013-05-23 16:56:38.449 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:56:38.449 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4f20 - 1,2,3 [ CSSD]2013-05-23 16:56:38.449 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3ea0) proc(0x22f4f20) pid(3617/3617) version 11:1:1:4 [ CSSD]2013-05-23 16:56:38.465 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f4f20), client(1/0x22ec760) [ CSSD]2013-05-23 16:56:38.465 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ec760) [ CSSD]2013-05-23 16:56:38.465 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:56:38.465 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 16:56:38.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:38.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:39.923 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:40.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:40.796 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:40.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:40.849 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014905, gin 1, member 0 [ CSSD]2013-05-23 16:56:40.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:56:40.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:56:40.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:40.856 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:56:40.856 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:56:41.928 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:42.803 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:42.803 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:42.831 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:42.831 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:43.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:44.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:44.834 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:45.959 [1191196992] >TRACE: clssgmAllocProc: (0x2318690) allocated [ CSSD]2013-05-23 16:56:45.961 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:56:45.961 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318690 - 1,2,3 [ CSSD]2013-05-23 16:56:45.961 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecd90) proc(0x2318690) pid(3857) version 11:1:1:4 [ CSSD]2013-05-23 16:56:45.978 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318690), client(1/0x2318de0) [ CSSD]2013-05-23 16:56:45.978 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:56:45.978 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318de0 with con 0x23188f0, requested num -1 [ CSSD]2013-05-23 16:56:45.978 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:45.978 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2008 (0x7f431bffdc10) [ CSSD]2013-05-23 16:56:45.978 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdc10 (RPC#2008) tag(7d8002c) sent to node 1 [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07d8002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014906, gin 1, member 0 [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318f50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdc10 (RPC#2008) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffdc10, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318de0 [ CSSD]2013-05-23 16:56:45.979 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2008 [ CSSD]2013-05-23 16:56:45.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:45.986 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:56:45.986 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318de0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:45.986 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318590) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:56:45.986 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2009 (0x7f431bffdc90) [ CSSD]2013-05-23 16:56:45.986 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdc90 (RPC#2009) tag(7d9002c) sent to node 1 [ CSSD]2013-05-23 16:56:45.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07d9002c) incarnation(174631051) [ CSSD]2013-05-23 16:56:45.986 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:56:45.986 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdc90 (RPC#2009) state 6, flags 0x100 [ CSSD]2013-05-23 16:56:45.986 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffdc90, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:56:45.986 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2009 [ CSSD]2013-05-23 16:56:45.986 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318590) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:56:46.189 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318690 [ CSSD]2013-05-23 16:56:46.189 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318690) con(0x22ecd90) skgpid ospid 3857 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:56:46.682 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:56:46.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2010 (0x7f431bffdd10) [ CSSD]2013-05-23 16:56:46.682 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdd10 (RPC#2010) tag(7da002c) sent to node 1 [ CSSD]2013-05-23 16:56:46.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:46.683 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2010 (0x7f431bffdd10) [ CSSD]2013-05-23 16:56:46.683 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdd10 (RPC#2010) state 6, flags 0x202 [ CSSD]2013-05-23 16:56:46.683 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffdd10, ret 0, client (nil) [ CSSD]2013-05-23 16:56:46.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2010 [ CSSD]2013-05-23 16:56:46.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:46.840 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:47.834 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:47.834 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:48.018 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:48.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:48.867 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:50.042 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:50.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:50.876 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:52.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:52.837 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:52.837 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:52.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:52.882 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:54.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:54.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:54.935 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:56.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:56.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014907, gin 1, member 0 [ CSSD]2013-05-23 16:56:56.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:56:56.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:56:56.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:56.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:56.109 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:56:56.109 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:56:56.706 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:56:56.706 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2011 (0x7f431bffdd90) [ CSSD]2013-05-23 16:56:56.706 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdd90 (RPC#2011) tag(7db002c) sent to node 1 [ CSSD]2013-05-23 16:56:56.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:56.707 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2011 (0x7f431bffdd90) [ CSSD]2013-05-23 16:56:56.707 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdd90 (RPC#2011) state 6, flags 0x202 [ CSSD]2013-05-23 16:56:56.707 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffdd90, ret 0, client (nil) [ CSSD]2013-05-23 16:56:56.707 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2011 [ CSSD]2013-05-23 16:56:56.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:56.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:56:57.839 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:56:57.840 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:56:58.198 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:56:58.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:56:58.988 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:00.204 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:01.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:01.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:01.221 [1191196992] >TRACE: clssgmAllocProc: (0x2318690) allocated [ CSSD]2013-05-23 16:57:01.222 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:57:01.222 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318690 - 1,2,3 [ CSSD]2013-05-23 16:57:01.222 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecd90) proc(0x2318690) pid(4015) version 11:1:1:4 [ CSSD]2013-05-23 16:57:01.228 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318690), client(1/0x2318de0) [ CSSD]2013-05-23 16:57:01.228 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:57:01.228 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318de0 with con 0x23188f0, requested num -1 [ CSSD]2013-05-23 16:57:01.228 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:01.228 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2012 (0x7f431bffde10) [ CSSD]2013-05-23 16:57:01.228 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffde10 (RPC#2012) tag(7dc002c) sent to node 1 [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07dc002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014908, gin 1, member 0 [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318f50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffde10 (RPC#2012) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffde10, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318de0 [ CSSD]2013-05-23 16:57:01.229 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2012 [ CSSD]2013-05-23 16:57:01.236 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:57:01.236 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318de0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:01.236 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318590) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:01.236 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2013 (0x7f431bffde90) [ CSSD]2013-05-23 16:57:01.236 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffde90 (RPC#2013) tag(7dd002c) sent to node 1 [ CSSD]2013-05-23 16:57:01.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07dd002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:01.236 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:01.236 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffde90 (RPC#2013) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:01.236 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffde90, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:57:01.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2013 [ CSSD]2013-05-23 16:57:01.237 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318590) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:57:01.439 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318690 [ CSSD]2013-05-23 16:57:01.439 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318690) con(0x22ecd90) skgpid ospid 4015 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:02.210 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:02.842 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:02.843 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:03.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:03.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:04.220 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:05.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:05.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:06.226 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:06.732 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:57:06.732 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2014 (0x7f431bffdf10) [ CSSD]2013-05-23 16:57:06.732 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdf10 (RPC#2014) tag(7de002c) sent to node 1 [ CSSD]2013-05-23 16:57:06.733 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:06.733 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2014 (0x7f431bffdf10) [ CSSD]2013-05-23 16:57:06.733 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdf10 (RPC#2014) state 6, flags 0x202 [ CSSD]2013-05-23 16:57:06.733 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffdf10, ret 0, client (nil) [ CSSD]2013-05-23 16:57:06.733 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2014 [ CSSD]2013-05-23 16:57:07.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:07.100 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:07.845 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:07.846 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:08.279 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:09.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:09.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:10.304 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:11.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:11.139 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:11.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:11.368 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014909, gin 1, member 0 [ CSSD]2013-05-23 16:57:11.368 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:11.368 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:11.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:11.376 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:11.376 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:11.848 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:11.848 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:57:12.309 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:13.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:13.184 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:13.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:13.254 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014910, gin 1, member 0 [ CSSD]2013-05-23 16:57:13.254 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:13.254 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:13.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:13.261 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:13.261 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:14.321 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:15.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:15.257 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:16.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:16.470 [1191196992] >TRACE: clssgmAllocProc: (0x2318760) allocated [ CSSD]2013-05-23 16:57:16.471 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:57:16.472 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318760 - 1,2,3 [ CSSD]2013-05-23 16:57:16.472 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ece60) proc(0x2318760) pid(4385) version 11:1:1:4 [ CSSD]2013-05-23 16:57:16.477 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318760), client(1/0x2318eb0) [ CSSD]2013-05-23 16:57:16.477 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318eb0) [ CSSD]2013-05-23 16:57:16.477 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318eb0 with con 0x23189c0, requested num -1 [ CSSD]2013-05-23 16:57:16.477 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:16.477 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2015 (0x7f431bffdf90) [ CSSD]2013-05-23 16:57:16.478 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffdf90 (RPC#2015) tag(7df002c) sent to node 1 [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07df002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014911, gin 1, member 0 [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2319020/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffdf90 (RPC#2015) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffdf90, ret 0, client 0x2318eb0 member 0x2318660 [ CSSD]2013-05-23 16:57:16.478 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318eb0 [ CSSD]2013-05-23 16:57:16.479 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2015 [ CSSD]2013-05-23 16:57:16.485 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318eb0) [ CSSD]2013-05-23 16:57:16.485 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318eb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:16.485 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:16.485 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2016 (0x7f431bffe010) [ CSSD]2013-05-23 16:57:16.485 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe010 (RPC#2016) tag(7e0002c) sent to node 1 [ CSSD]2013-05-23 16:57:16.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07e0002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:16.486 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:16.486 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe010 (RPC#2016) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:16.486 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe010, ret 0, client 0x2318eb0 member 0x2318660 [ CSSD]2013-05-23 16:57:16.486 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2016 [ CSSD]2013-05-23 16:57:16.486 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:57:16.689 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318760 [ CSSD]2013-05-23 16:57:16.689 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318760) con(0x22ece60) skgpid ospid 4385 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:16.756 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:57:16.756 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2017 (0x7f431bffe090) [ CSSD]2013-05-23 16:57:16.756 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe090 (RPC#2017) tag(7e1002c) sent to node 1 [ CSSD]2013-05-23 16:57:16.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:16.757 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2017 (0x7f431bffe090) [ CSSD]2013-05-23 16:57:16.757 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe090 (RPC#2017) state 6, flags 0x202 [ CSSD]2013-05-23 16:57:16.757 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe090, ret 0, client (nil) [ CSSD]2013-05-23 16:57:16.757 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2017 [ CSSD]2013-05-23 16:57:16.850 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:16.851 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:17.263 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:17.263 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:17.981 [1191196992] >TRACE: clssgmAllocProc: (0x2318690) allocated [ CSSD]2013-05-23 16:57:17.982 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:57:17.983 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318690 - 1,2,3 [ CSSD]2013-05-23 16:57:17.983 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecd90) proc(0x2318690) pid(4436) version 11:1:1:4 [ CSSD]2013-05-23 16:57:18.107 [1191196992] >TRACE: clssgmAllocProc: (0x22e4ef0) allocated [ CSSD]2013-05-23 16:57:18.108 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:57:18.108 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4ef0 - 1,2,3 [ CSSD]2013-05-23 16:57:18.108 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2318e60) proc(0x22e4ef0) pid(4446) version 11:1:1:4 [ CSSD]2013-05-23 16:57:18.124 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:57:18.126 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:57:18.128 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4ef0 [ CSSD]2013-05-23 16:57:18.128 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4ef0) con(0x2318e60) skgpid ospid 4446 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:18.209 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(74/0x2318ac0) [ CSSD]2013-05-23 16:57:18.209 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 74 (0x2318ac0) [ CSSD]2013-05-23 16:57:18.209 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x2318ac0 with con 0x22e51e0, requested num -1 [ CSSD]2013-05-23 16:57:18.209 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:57:18.209 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2018 (0x7f431bffe110) [ CSSD]2013-05-23 16:57:18.209 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe110 (RPC#2018) tag(7e2002c) sent to node 1 [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07e2002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014912, gin 1, member 0 [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x2318c30/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe110 (RPC#2018) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffe110, ret 0, client 0x2318ac0 member 0x2318570 [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318ac0 [ CSSD]2013-05-23 16:57:18.210 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2018 [ CSSD]2013-05-23 16:57:18.218 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 74 (0x2318ac0) [ CSSD]2013-05-23 16:57:18.218 [1191196992] >TRACE: clssgmExitGrock: client 74 (0x2318ac0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:57:18.218 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318570) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 16:57:18.218 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2019 (0x7f431bffe190) [ CSSD]2013-05-23 16:57:18.218 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe190 (RPC#2019) tag(7e3002c) sent to node 1 [ CSSD]2013-05-23 16:57:18.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07e3002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:18.219 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 16:57:18.219 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe190 (RPC#2019) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:18.219 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe190, ret 0, client 0x2318ac0 member 0x2318570 [ CSSD]2013-05-23 16:57:18.219 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2019 [ CSSD]2013-05-23 16:57:18.219 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x2318570) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:57:18.220 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318690 [ CSSD]2013-05-23 16:57:18.220 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318690) con(0x22ecd90) skgpid ospid 4436 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:18.379 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:19.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:19.274 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:20.405 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:21.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:21.280 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:21.854 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:21.854 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:22.410 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:23.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:23.306 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:24.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:24.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:24.970 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014913, gin 1, member 0 [ CSSD]2013-05-23 16:57:24.970 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:24.970 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:24.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:24.978 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:24.978 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:25.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:25.357 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:26.427 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:26.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:26.625 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014914, gin 1, member 0 [ CSSD]2013-05-23 16:57:26.625 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:26.625 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:26.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:26.632 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:26.632 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:26.780 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:57:26.780 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2020 (0x7f431bffe210) [ CSSD]2013-05-23 16:57:26.780 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe210 (RPC#2020) tag(7e4002c) sent to node 1 [ CSSD]2013-05-23 16:57:26.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:26.781 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2020 (0x7f431bffe210) [ CSSD]2013-05-23 16:57:26.781 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe210 (RPC#2020) state 6, flags 0x202 [ CSSD]2013-05-23 16:57:26.781 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe210, ret 0, client (nil) [ CSSD]2013-05-23 16:57:26.781 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2020 [ CSSD]2013-05-23 16:57:26.857 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:26.857 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:27.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:27.363 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:28.450 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:28.794 [1191196992] >TRACE: clssgmAllocProc: (0x2318760) allocated [ CSSD]2013-05-23 16:57:28.794 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:57:28.795 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318760 - 1,2,3 [ CSSD]2013-05-23 16:57:28.795 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ece60) proc(0x2318760) pid(4547) version 11:1:1:4 [ CSSD]2013-05-23 16:57:28.812 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318760), client(1/0x2318f20) [ CSSD]2013-05-23 16:57:28.812 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318f20) [ CSSD]2013-05-23 16:57:28.812 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318f20 with con 0x2318a30, requested num -1 [ CSSD]2013-05-23 16:57:28.812 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:28.812 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2021 (0x7f431bffe290) [ CSSD]2013-05-23 16:57:28.812 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe290 (RPC#2021) tag(7e5002c) sent to node 1 [ CSSD]2013-05-23 16:57:28.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07e5002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:28.812 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014915, gin 1, member 0 [ CSSD]2013-05-23 16:57:28.812 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2319090/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:57:28.813 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:57:28.813 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:57:28.813 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe290 (RPC#2021) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:28.813 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffe290, ret 0, client 0x2318f20 member 0x2318660 [ CSSD]2013-05-23 16:57:28.813 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318f20 [ CSSD]2013-05-23 16:57:28.813 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2021 [ CSSD]2013-05-23 16:57:28.819 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318f20) [ CSSD]2013-05-23 16:57:28.819 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318f20), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:28.819 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:28.819 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2022 (0x7f431bffe310) [ CSSD]2013-05-23 16:57:28.819 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe310 (RPC#2022) tag(7e6002c) sent to node 1 [ CSSD]2013-05-23 16:57:28.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07e6002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:28.820 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:28.820 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe310 (RPC#2022) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:28.820 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe310, ret 0, client 0x2318f20 member 0x2318660 [ CSSD]2013-05-23 16:57:28.820 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2022 [ CSSD]2013-05-23 16:57:28.820 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:57:29.022 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318760 [ CSSD]2013-05-23 16:57:29.022 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318760) con(0x22ece60) skgpid ospid 4547 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:29.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:29.374 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:30.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:31.379 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:31.379 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:31.724 [1191196992] >TRACE: clssgmAllocProc: (0x2318690) allocated [ CSSD]2013-05-23 16:57:31.725 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:57:31.725 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318690 - 1,2,3 [ CSSD]2013-05-23 16:57:31.725 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecd90) proc(0x2318690) pid(4584) version 11:1:1:4 [ CSSD]2013-05-23 16:57:31.731 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318690), client(1/0x2318de0) [ CSSD]2013-05-23 16:57:31.731 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:57:31.731 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318de0 with con 0x23188f0, requested num -1 [ CSSD]2013-05-23 16:57:31.731 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:31.731 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2023 (0x7f431bffe390) [ CSSD]2013-05-23 16:57:31.731 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe390 (RPC#2023) tag(7e7002c) sent to node 1 [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07e7002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014916, gin 1, member 0 [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318f50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe390 (RPC#2023) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffe390, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318de0 [ CSSD]2013-05-23 16:57:31.732 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2023 [ CSSD]2013-05-23 16:57:31.739 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:57:31.739 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318de0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:31.739 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318590) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:31.739 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2024 (0x7f431bffe410) [ CSSD]2013-05-23 16:57:31.739 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe410 (RPC#2024) tag(7e8002c) sent to node 1 [ CSSD]2013-05-23 16:57:31.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07e8002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:31.740 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:31.740 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe410 (RPC#2024) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:31.740 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe410, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:57:31.740 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2024 [ CSSD]2013-05-23 16:57:31.740 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318590) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:57:31.860 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:31.860 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:31.942 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318690 [ CSSD]2013-05-23 16:57:31.942 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318690) con(0x22ecd90) skgpid ospid 4584 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:32.462 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:33.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:33.404 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:34.492 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:35.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:35.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:36.537 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:36.803 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:57:36.803 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2025 (0x7f431bffe490) [ CSSD]2013-05-23 16:57:36.803 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe490 (RPC#2025) tag(7e9002c) sent to node 1 [ CSSD]2013-05-23 16:57:36.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:36.804 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2025 (0x7f431bffe490) [ CSSD]2013-05-23 16:57:36.804 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe490 (RPC#2025) state 6, flags 0x202 [ CSSD]2013-05-23 16:57:36.804 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe490, ret 0, client (nil) [ CSSD]2013-05-23 16:57:36.804 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2025 [ CSSD]2013-05-23 16:57:36.863 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:36.863 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:37.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:37.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:38.590 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:39.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:39.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:40.616 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:41.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:41.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:41.866 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:41.866 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:41.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:41.887 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014917, gin 1, member 0 [ CSSD]2013-05-23 16:57:41.887 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:41.887 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:41.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:41.894 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:41.894 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:42.622 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:42.792 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:42.792 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014918, gin 1, member 0 [ CSSD]2013-05-23 16:57:42.792 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:42.792 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:42.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:42.799 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:42.799 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:43.484 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:43.484 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:44.632 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:45.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:45.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:46.638 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:46.826 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:57:46.826 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2026 (0x7f431bffe510) [ CSSD]2013-05-23 16:57:46.826 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe510 (RPC#2026) tag(7ea002c) sent to node 1 [ CSSD]2013-05-23 16:57:46.827 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:46.827 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2026 (0x7f431bffe510) [ CSSD]2013-05-23 16:57:46.827 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe510 (RPC#2026) state 6, flags 0x202 [ CSSD]2013-05-23 16:57:46.827 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe510, ret 0, client (nil) [ CSSD]2013-05-23 16:57:46.827 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2026 [ CSSD]2013-05-23 16:57:46.869 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:46.869 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:46.974 [1191196992] >TRACE: clssgmAllocProc: (0x2318690) allocated [ CSSD]2013-05-23 16:57:46.974 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:57:46.975 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318690 - 1,2,3 [ CSSD]2013-05-23 16:57:46.975 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecd90) proc(0x2318690) pid(4779) version 11:1:1:4 [ CSSD]2013-05-23 16:57:46.981 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318690), client(1/0x2318de0) [ CSSD]2013-05-23 16:57:46.981 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:57:46.981 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318de0 with con 0x23188f0, requested num -1 [ CSSD]2013-05-23 16:57:46.981 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:46.981 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2027 (0x7f431bffe590) [ CSSD]2013-05-23 16:57:46.981 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe590 (RPC#2027) tag(7eb002c) sent to node 1 [ CSSD]2013-05-23 16:57:46.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07eb002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:46.981 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014919, gin 1, member 0 [ CSSD]2013-05-23 16:57:46.981 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2318f50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:57:46.982 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:57:46.982 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:57:46.982 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe590 (RPC#2027) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:46.982 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffe590, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:57:46.982 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318de0 [ CSSD]2013-05-23 16:57:46.982 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2027 [ CSSD]2013-05-23 16:57:46.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318de0) [ CSSD]2013-05-23 16:57:46.989 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318de0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:46.989 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318590) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:57:46.989 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2028 (0x7f431bffe610) [ CSSD]2013-05-23 16:57:46.989 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe610 (RPC#2028) tag(7ec002c) sent to node 1 [ CSSD]2013-05-23 16:57:46.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07ec002c) incarnation(174631051) [ CSSD]2013-05-23 16:57:46.989 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:57:46.989 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe610 (RPC#2028) state 6, flags 0x100 [ CSSD]2013-05-23 16:57:46.989 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe610, ret 0, client 0x2318de0 member 0x2318590 [ CSSD]2013-05-23 16:57:46.990 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2028 [ CSSD]2013-05-23 16:57:46.990 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318590) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:57:47.192 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318690 [ CSSD]2013-05-23 16:57:47.192 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318690) con(0x22ecd90) skgpid ospid 4779 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:57:47.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:47.545 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:48.691 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:49.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:49.555 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:50.697 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:51.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:51.581 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:51.873 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:51.873 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:52.703 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:53.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:53.587 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:54.713 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:55.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:55.638 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:56.720 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:56.851 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:57:56.852 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2029 (0x7f431bffe690) [ CSSD]2013-05-23 16:57:56.852 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe690 (RPC#2029) tag(7ed002c) sent to node 1 [ CSSD]2013-05-23 16:57:56.852 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:56.852 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2029 (0x7f431bffe690) [ CSSD]2013-05-23 16:57:56.852 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe690 (RPC#2029) state 6, flags 0x202 [ CSSD]2013-05-23 16:57:56.852 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe690, ret 0, client (nil) [ CSSD]2013-05-23 16:57:56.853 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2029 [ CSSD]2013-05-23 16:57:56.876 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:57:56.876 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:57:57.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:57.133 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014920, gin 1, member 0 [ CSSD]2013-05-23 16:57:57.133 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22ec8d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:57:57.133 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:57:57.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:57.140 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:57:57.140 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22ec660) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:57:57.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:57.645 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:57:58.812 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:57:59.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:57:59.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:00.474 [1191196992] >TRACE: clssgmAllocProc: (0x2318760) allocated [ CSSD]2013-05-23 16:58:00.475 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:58:00.475 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318760 - 1,2,3 [ CSSD]2013-05-23 16:58:00.475 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ecd90) proc(0x2318760) pid(4846/4846) version 11:1:1:4 [ CSSD]2013-05-23 16:58:00.491 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318760), client(1/0x2318d30) [ CSSD]2013-05-23 16:58:00.492 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2318d30) [ CSSD]2013-05-23 16:58:00.492 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:58:00.492 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 16:58:00.810 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x2318760), client(2/0x2319210) [ CSSD]2013-05-23 16:58:00.810 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2319210) [ CSSD]2013-05-23 16:58:00.810 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 16:58:00.810 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 16:58:00.837 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:01.681 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:01.681 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:01.879 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:01.879 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:02.223 [1191196992] >TRACE: clssgmAllocProc: (0x22e5030) allocated [ CSSD]2013-05-23 16:58:02.224 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:58:02.224 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5030 - 1,2,3 [ CSSD]2013-05-23 16:58:02.224 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2319380) proc(0x22e5030) pid(4937) version 11:1:1:4 [ CSSD]2013-05-23 16:58:02.230 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e5030), client(1/0x2321ad0) [ CSSD]2013-05-23 16:58:02.230 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 16:58:02.230 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321ad0 with con 0x2319520, requested num -1 [ CSSD]2013-05-23 16:58:02.230 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:02.230 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2030 (0x7f431bffe710) [ CSSD]2013-05-23 16:58:02.230 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe710 (RPC#2030) tag(7ee002c) sent to node 1 [ CSSD]2013-05-23 16:58:02.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07ee002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:02.230 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014921, gin 1, member 0 [ CSSD]2013-05-23 16:58:02.230 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:58:02.231 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:58:02.231 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:58:02.231 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe710 (RPC#2030) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:02.231 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffe710, ret 0, client 0x2321ad0 member 0x22e4f00 [ CSSD]2013-05-23 16:58:02.231 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ad0 [ CSSD]2013-05-23 16:58:02.231 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2030 [ CSSD]2013-05-23 16:58:02.237 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 16:58:02.237 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321ad0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:02.237 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4f00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:02.237 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2031 (0x7f431bffe790) [ CSSD]2013-05-23 16:58:02.237 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe790 (RPC#2031) tag(7ef002c) sent to node 1 [ CSSD]2013-05-23 16:58:02.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07ef002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:02.237 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:02.237 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe790 (RPC#2031) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:02.237 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe790, ret 0, client 0x2321ad0 member 0x22e4f00 [ CSSD]2013-05-23 16:58:02.238 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2031 [ CSSD]2013-05-23 16:58:02.238 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4f00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:58:02.440 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5030 [ CSSD]2013-05-23 16:58:02.440 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5030) con(0x2319380) skgpid ospid 4937 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:58:02.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:03.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:03.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:04.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:05.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:05.708 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:06.880 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:58:06.880 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2032 (0x7f431bffe810) [ CSSD]2013-05-23 16:58:06.880 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe810 (RPC#2032) tag(7f0002c) sent to node 1 [ CSSD]2013-05-23 16:58:06.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:06.881 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2032 (0x7f431bffe810) [ CSSD]2013-05-23 16:58:06.881 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe810 (RPC#2032) state 6, flags 0x202 [ CSSD]2013-05-23 16:58:06.881 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe810, ret 0, client (nil) [ CSSD]2013-05-23 16:58:06.881 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2032 [ CSSD]2013-05-23 16:58:06.883 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:06.883 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:06.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:07.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:07.715 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:08.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:09.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:09.736 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:10.917 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:11.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:11.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:11.886 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:11.886 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:12.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:12.395 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014922, gin 1, member 0 [ CSSD]2013-05-23 16:58:12.395 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:58:12.395 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:58:12.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:12.402 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:58:12.402 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319110) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:58:12.930 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:13.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:13.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:14.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:15.760 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:15.760 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:16.889 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:16.889 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:16.904 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:58:16.904 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2033 (0x7f431bffe890) [ CSSD]2013-05-23 16:58:16.904 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe890 (RPC#2033) tag(7f1002c) sent to node 1 [ CSSD]2013-05-23 16:58:16.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:16.905 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2033 (0x7f431bffe890) [ CSSD]2013-05-23 16:58:16.905 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe890 (RPC#2033) state 6, flags 0x202 [ CSSD]2013-05-23 16:58:16.905 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffe890, ret 0, client (nil) [ CSSD]2013-05-23 16:58:16.905 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2033 [ CSSD]2013-05-23 16:58:16.961 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:17.471 [1191196992] >TRACE: clssgmAllocProc: (0x22e50c0) allocated [ CSSD]2013-05-23 16:58:17.472 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:58:17.473 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e50c0 - 1,2,3 [ CSSD]2013-05-23 16:58:17.473 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e37a0) proc(0x22e50c0) pid(5106) version 11:1:1:4 [ CSSD]2013-05-23 16:58:17.479 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e50c0), client(1/0x2321c50) [ CSSD]2013-05-23 16:58:17.479 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321c50) [ CSSD]2013-05-23 16:58:17.479 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321c50 with con 0x22e4ec0, requested num -1 [ CSSD]2013-05-23 16:58:17.479 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:17.479 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2034 (0x7f431bffe910) [ CSSD]2013-05-23 16:58:17.479 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe910 (RPC#2034) tag(7f2002c) sent to node 1 [ CSSD]2013-05-23 16:58:17.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07f2002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:17.479 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014923, gin 1, member 0 [ CSSD]2013-05-23 16:58:17.479 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321dc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:58:17.480 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:58:17.480 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:58:17.480 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe910 (RPC#2034) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:17.480 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffe910, ret 0, client 0x2321c50 member 0x23220c0 [ CSSD]2013-05-23 16:58:17.480 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321c50 [ CSSD]2013-05-23 16:58:17.480 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2034 [ CSSD]2013-05-23 16:58:17.487 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321c50) [ CSSD]2013-05-23 16:58:17.487 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321c50), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:17.487 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23220c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:17.487 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2035 (0x7f431bffe990) [ CSSD]2013-05-23 16:58:17.487 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffe990 (RPC#2035) tag(7f3002c) sent to node 1 [ CSSD]2013-05-23 16:58:17.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07f3002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:17.487 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:17.487 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffe990 (RPC#2035) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:17.487 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffe990, ret 0, client 0x2321c50 member 0x23220c0 [ CSSD]2013-05-23 16:58:17.487 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2035 [ CSSD]2013-05-23 16:58:17.487 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23220c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:58:17.690 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e50c0 [ CSSD]2013-05-23 16:58:17.690 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e50c0) con(0x22e37a0) skgpid ospid 5106 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:58:17.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:17.786 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:18.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:19.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:19.798 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:21.038 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:21.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:21.804 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:21.892 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:21.892 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:23.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:23.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:23.810 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:25.155 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:25.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:25.241 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014924, gin 1, member 0 [ CSSD]2013-05-23 16:58:25.241 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:58:25.241 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:58:25.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:25.249 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:58:25.249 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319380) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:58:25.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:25.861 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:26.895 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:26.896 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:26.927 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:58:26.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2036 (0x7f431bffea10) [ CSSD]2013-05-23 16:58:26.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffea10 (RPC#2036) tag(7f4002c) sent to node 1 [ CSSD]2013-05-23 16:58:26.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:26.928 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2036 (0x7f431bffea10) [ CSSD]2013-05-23 16:58:26.929 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffea10 (RPC#2036) state 6, flags 0x202 [ CSSD]2013-05-23 16:58:26.929 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffea10, ret 0, client (nil) [ CSSD]2013-05-23 16:58:26.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2036 [ CSSD]2013-05-23 16:58:27.161 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:27.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:27.654 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014925, gin 1, member 0 [ CSSD]2013-05-23 16:58:27.654 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:58:27.654 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:58:27.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:27.661 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:58:27.662 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319020) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:58:27.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:27.886 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:29.062 [1191196992] >TRACE: clssgmAllocProc: (0x22e5000) allocated [ CSSD]2013-05-23 16:58:29.064 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:58:29.064 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5000 - 1,2,3 [ CSSD]2013-05-23 16:58:29.064 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5000) pid(5489) version 11:1:1:4 [ CSSD]2013-05-23 16:58:29.070 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e5000), client(1/0x2319530) [ CSSD]2013-05-23 16:58:29.070 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:58:29.070 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319530 with con 0x22e5280, requested num -1 [ CSSD]2013-05-23 16:58:29.070 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:29.070 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2037 (0x7f431bffea90) [ CSSD]2013-05-23 16:58:29.071 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffea90 (RPC#2037) tag(7f5002c) sent to node 1 [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07f5002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014926, gin 1, member 0 [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffea90 (RPC#2037) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffea90, ret 0, client 0x2319530 member 0x22e4ef0 [ CSSD]2013-05-23 16:58:29.071 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319530 [ CSSD]2013-05-23 16:58:29.072 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2037 [ CSSD]2013-05-23 16:58:29.078 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:58:29.078 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319530), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:29.078 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:29.078 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2038 (0x7f431bffeb10) [ CSSD]2013-05-23 16:58:29.078 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffeb10 (RPC#2038) tag(7f6002c) sent to node 1 [ CSSD]2013-05-23 16:58:29.078 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07f6002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:29.079 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:29.079 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffeb10 (RPC#2038) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:29.079 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffeb10, ret 0, client 0x2319530 member 0x22e4ef0 [ CSSD]2013-05-23 16:58:29.079 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2038 [ CSSD]2013-05-23 16:58:29.079 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:58:29.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:29.281 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5000 [ CSSD]2013-05-23 16:58:29.281 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5000) con(0x22e36e0) skgpid ospid 5489 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:58:29.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:29.957 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:31.199 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:31.899 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:31.899 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:31.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:31.964 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:32.722 [1191196992] >TRACE: clssgmAllocProc: (0x22e4dc0) allocated [ CSSD]2013-05-23 16:58:32.723 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:58:32.724 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4dc0 - 1,2,3 [ CSSD]2013-05-23 16:58:32.724 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e5220) proc(0x22e4dc0) pid(5519) version 11:1:1:4 [ CSSD]2013-05-23 16:58:32.730 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4dc0), client(1/0x22e5460) [ CSSD]2013-05-23 16:58:32.730 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:58:32.730 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e4bc0, requested num -1 [ CSSD]2013-05-23 16:58:32.730 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:32.730 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2039 (0x7f431bffeb90) [ CSSD]2013-05-23 16:58:32.730 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffeb90 (RPC#2039) tag(7f7002c) sent to node 1 [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07f7002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014927, gin 1, member 0 [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffeb90 (RPC#2039) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffeb90, ret 0, client 0x22e5460 member 0x22e5130 [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:58:32.731 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2039 [ CSSD]2013-05-23 16:58:32.738 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:58:32.738 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:32.738 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e5130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:32.739 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2040 (0x7f431bffec10) [ CSSD]2013-05-23 16:58:32.739 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffec10 (RPC#2040) tag(7f8002c) sent to node 1 [ CSSD]2013-05-23 16:58:32.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07f8002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:32.739 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:32.739 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffec10 (RPC#2040) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:32.739 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffec10, ret 0, client 0x22e5460 member 0x22e5130 [ CSSD]2013-05-23 16:58:32.739 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2040 [ CSSD]2013-05-23 16:58:32.739 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e5130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:58:32.942 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4dc0 [ CSSD]2013-05-23 16:58:32.942 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4dc0) con(0x22e5220) skgpid ospid 5519 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:58:33.211 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:33.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:33.970 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:35.217 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:35.901 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:35.901 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:58:36.000 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:36.000 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:36.953 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:58:36.954 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2041 (0x7f431bffec90) [ CSSD]2013-05-23 16:58:36.954 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffec90 (RPC#2041) tag(7f9002c) sent to node 1 [ CSSD]2013-05-23 16:58:36.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:36.955 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2041 (0x7f431bffec90) [ CSSD]2013-05-23 16:58:36.955 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffec90 (RPC#2041) state 6, flags 0x202 [ CSSD]2013-05-23 16:58:36.955 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffec90, ret 0, client (nil) [ CSSD]2013-05-23 16:58:36.955 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2041 [ CSSD]2013-05-23 16:58:37.222 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:38.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:38.006 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:39.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:40.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:40.059 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:40.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:40.762 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014928, gin 1, member 0 [ CSSD]2013-05-23 16:58:40.762 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 16:58:40.762 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:58:40.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:40.770 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 16:58:40.770 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x2319110) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:58:40.904 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:40.905 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:40.905 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:58:40.905 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:58:40.907 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:58:40.908 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:58:40.909 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:58:40.910 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:58:41.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:42.084 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:42.084 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:42.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:42.905 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014929, gin 1, member 0 [ CSSD]2013-05-23 16:58:42.905 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:58:42.905 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:58:42.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:42.912 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:58:42.912 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319110) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:58:43.301 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:44.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:44.090 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:45.308 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:45.907 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:45.908 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:46.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:46.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:46.977 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:58:46.978 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2042 (0x7f431bffed10) [ CSSD]2013-05-23 16:58:46.978 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffed10 (RPC#2042) tag(7fa002c) sent to node 1 [ CSSD]2013-05-23 16:58:46.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:46.978 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2042 (0x7f431bffed10) [ CSSD]2013-05-23 16:58:46.978 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffed10 (RPC#2042) state 6, flags 0x202 [ CSSD]2013-05-23 16:58:46.978 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffed10, ret 0, client (nil) [ CSSD]2013-05-23 16:58:46.978 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2042 [ CSSD]2013-05-23 16:58:47.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:47.973 [1191196992] >TRACE: clssgmAllocProc: (0x22e50c0) allocated [ CSSD]2013-05-23 16:58:47.975 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:58:47.975 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e50c0 - 1,2,3 [ CSSD]2013-05-23 16:58:47.975 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e37a0) proc(0x22e50c0) pid(5671) version 11:1:1:4 [ CSSD]2013-05-23 16:58:47.981 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e50c0), client(1/0x2321c50) [ CSSD]2013-05-23 16:58:47.981 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321c50) [ CSSD]2013-05-23 16:58:47.981 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321c50 with con 0x22e4ec0, requested num -1 [ CSSD]2013-05-23 16:58:47.981 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:47.981 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2043 (0x7f431bffed90) [ CSSD]2013-05-23 16:58:47.982 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffed90 (RPC#2043) tag(7fb002c) sent to node 1 [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07fb002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014930, gin 1, member 0 [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321dc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffed90 (RPC#2043) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffed90, ret 0, client 0x2321c50 member 0x23220c0 [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321c50 [ CSSD]2013-05-23 16:58:47.982 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2043 [ CSSD]2013-05-23 16:58:47.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321c50) [ CSSD]2013-05-23 16:58:47.989 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321c50), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:47.989 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23220c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:58:47.989 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2044 (0x7f431bffee10) [ CSSD]2013-05-23 16:58:47.989 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffee10 (RPC#2044) tag(7fc002c) sent to node 1 [ CSSD]2013-05-23 16:58:47.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07fc002c) incarnation(174631051) [ CSSD]2013-05-23 16:58:47.989 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:58:47.989 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffee10 (RPC#2044) state 6, flags 0x100 [ CSSD]2013-05-23 16:58:47.989 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffee10, ret 0, client 0x2321c50 member 0x23220c0 [ CSSD]2013-05-23 16:58:47.990 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2044 [ CSSD]2013-05-23 16:58:47.990 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23220c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:58:48.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:48.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:48.192 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e50c0 [ CSSD]2013-05-23 16:58:48.192 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e50c0) con(0x22e37a0) skgpid ospid 5671 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:58:49.323 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:50.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:50.159 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:50.911 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:50.911 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:58:51.348 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:52.165 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:52.165 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:53.402 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:54.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:54.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:54.913 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:54.914 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:58:55.408 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:56.182 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:56.182 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:57.037 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:58:57.038 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2045 (0x7f431bffee90) [ CSSD]2013-05-23 16:58:57.038 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffee90 (RPC#2045) tag(7fd002c) sent to node 1 [ CSSD]2013-05-23 16:58:57.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:57.038 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 2045 (0x7f431bffee90) [ CSSD]2013-05-23 16:58:57.038 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffee90 (RPC#2045) state 6, flags 0x202 [ CSSD]2013-05-23 16:58:57.038 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bffee90, ret 0, client (nil) [ CSSD]2013-05-23 16:58:57.038 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2045 [ CSSD]2013-05-23 16:58:57.414 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:58.155 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:58.155 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014931, gin 1, member 0 [ CSSD]2013-05-23 16:58:58.155 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:58:58.155 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:58:58.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:58.162 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:58:58.162 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319020) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:58:58.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:58:58.207 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:58:59.424 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:58:59.916 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:58:59.917 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:00.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:00.260 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:01.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:02.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:02.285 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:03.224 [1191196992] >TRACE: clssgmAllocProc: (0x22e5040) allocated [ CSSD]2013-05-23 16:59:03.226 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:59:03.226 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5040 - 1,2,3 [ CSSD]2013-05-23 16:59:03.226 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5040) pid(5837) version 11:1:1:4 [ CSSD]2013-05-23 16:59:03.232 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e5040), client(1/0x2319530) [ CSSD]2013-05-23 16:59:03.232 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:03.232 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319530 with con 0x22e5260, requested num -1 [ CSSD]2013-05-23 16:59:03.232 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:03.232 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2046 (0x7f431bffef10) [ CSSD]2013-05-23 16:59:03.232 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffef10 (RPC#2046) tag(7fe002c) sent to node 1 [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(07fe002c) incarnation(174631051) [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014932, gin 1, member 0 [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffef10 (RPC#2046) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bffef10, ret 0, client 0x2319530 member 0x22e4f30 [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319530 [ CSSD]2013-05-23 16:59:03.233 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2046 [ CSSD]2013-05-23 16:59:03.240 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:03.240 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319530), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:03.240 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4f30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:03.240 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2047 (0x7f431bffef90) [ CSSD]2013-05-23 16:59:03.240 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bffef90 (RPC#2047) tag(7ff002c) sent to node 1 [ CSSD]2013-05-23 16:59:03.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(07ff002c) incarnation(174631051) [ CSSD]2013-05-23 16:59:03.241 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:03.241 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bffef90 (RPC#2047) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:03.241 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bffef90, ret 0, client 0x2319530 member 0x22e4f30 [ CSSD]2013-05-23 16:59:03.241 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2047 [ CSSD]2013-05-23 16:59:03.241 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4f30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:59:03.443 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5040 [ CSSD]2013-05-23 16:59:03.443 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5040) con(0x22e36e0) skgpid ospid 5837 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:59:03.502 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:04.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:04.296 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:04.920 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:04.920 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:05.528 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:06.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:06.302 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:07.063 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:59:07.063 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 0 (0x7f431bfbf010) [ CSSD]2013-05-23 16:59:07.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf010 (RPC#0) tag(2d) sent to node 1 [ CSSD]2013-05-23 16:59:07.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:07.064 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 0 (0x7f431bfbf010) [ CSSD]2013-05-23 16:59:07.064 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf010 (RPC#0) state 6, flags 0x202 [ CSSD]2013-05-23 16:59:07.064 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf010, ret 0, client (nil) [ CSSD]2013-05-23 16:59:07.064 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 0 [ CSSD]2013-05-23 16:59:07.533 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:08.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:08.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:09.544 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:09.923 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:09.923 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:10.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:10.360 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:11.550 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:12.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:12.385 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:13.405 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:13.405 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014933, gin 1, member 0 [ CSSD]2013-05-23 16:59:13.405 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:59:13.405 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:59:13.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:13.412 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:59:13.412 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319110) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:59:13.603 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:13.926 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:13.926 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:59:14.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:14.398 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:15.609 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:16.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:16.423 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:17.092 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:59:17.092 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1 (0x7f431bfbf090) [ CSSD]2013-05-23 16:59:17.093 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf090 (RPC#1) tag(1002d) sent to node 1 [ CSSD]2013-05-23 16:59:17.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:17.093 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1 (0x7f431bfbf090) [ CSSD]2013-05-23 16:59:17.093 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf090 (RPC#1) state 6, flags 0x202 [ CSSD]2013-05-23 16:59:17.093 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf090, ret 0, client (nil) [ CSSD]2013-05-23 16:59:17.093 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1 [ CSSD]2013-05-23 16:59:17.614 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:18.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:18.428 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:18.488 [1191196992] >TRACE: clssgmAllocProc: (0x22e5000) allocated [ CSSD]2013-05-23 16:59:18.489 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:59:18.489 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5000 - 1,2,3 [ CSSD]2013-05-23 16:59:18.489 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5000) pid(6011) version 11:1:1:4 [ CSSD]2013-05-23 16:59:18.495 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e5000), client(1/0x2319530) [ CSSD]2013-05-23 16:59:18.495 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:18.495 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319530 with con 0x22e5280, requested num -1 [ CSSD]2013-05-23 16:59:18.495 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:18.495 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 2 (0x7f431bfbf110) [ CSSD]2013-05-23 16:59:18.495 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf110 (RPC#2) tag(2002d) sent to node 1 [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0002002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014934, gin 1, member 0 [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf110 (RPC#2) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbf110, ret 0, client 0x2319530 member 0x22e4ef0 [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319530 [ CSSD]2013-05-23 16:59:18.496 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 2 [ CSSD]2013-05-23 16:59:18.502 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:18.502 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319530), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:18.502 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:18.502 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 3 (0x7f431bfbf190) [ CSSD]2013-05-23 16:59:18.502 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf190 (RPC#3) tag(3002d) sent to node 1 [ CSSD]2013-05-23 16:59:18.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0003002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:18.503 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:18.503 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf190 (RPC#3) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:18.503 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbf190, ret 0, client 0x2319530 member 0x22e4ef0 [ CSSD]2013-05-23 16:59:18.503 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 3 [ CSSD]2013-05-23 16:59:18.503 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:59:18.705 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5000 [ CSSD]2013-05-23 16:59:18.705 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5000) con(0x22e36e0) skgpid ospid 6011 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:59:18.929 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:18.929 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:19.625 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:20.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:20.461 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:21.670 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:22.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:22.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:23.703 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:23.932 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:23.932 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:24.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:24.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:25.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:25.506 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014935, gin 1, member 0 [ CSSD]2013-05-23 16:59:25.506 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:59:25.506 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:59:25.512 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:25.512 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:59:25.512 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319110) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:59:25.709 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:26.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:26.472 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:27.116 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:59:27.117 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 4 (0x7f431bfbf210) [ CSSD]2013-05-23 16:59:27.117 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf210 (RPC#4) tag(4002d) sent to node 1 [ CSSD]2013-05-23 16:59:27.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:27.117 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 4 (0x7f431bfbf210) [ CSSD]2013-05-23 16:59:27.117 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf210 (RPC#4) state 6, flags 0x202 [ CSSD]2013-05-23 16:59:27.117 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf210, ret 0, client (nil) [ CSSD]2013-05-23 16:59:27.117 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 4 [ CSSD]2013-05-23 16:59:27.678 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:59:27.678 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:59:27.679 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:59:27.680 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 16:59:27.715 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:27.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:27.851 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014936, gin 1, member 0 [ CSSD]2013-05-23 16:59:27.851 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22e3300/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 16:59:27.851 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 16:59:27.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:27.859 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 16:59:27.859 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x2319380) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:59:28.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:28.478 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:28.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:28.672 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014937, gin 1, member 0 [ CSSD]2013-05-23 16:59:28.672 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:59:28.672 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:59:28.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:28.679 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:59:28.679 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319380) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:59:28.935 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:28.936 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:29.319 [1191196992] >TRACE: clssgmAllocProc: (0x22e5000) allocated [ CSSD]2013-05-23 16:59:29.320 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:59:29.321 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5000 - 1,2,3 [ CSSD]2013-05-23 16:59:29.321 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5000) pid(6130) version 11:1:1:4 [ CSSD]2013-05-23 16:59:29.327 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e5000), client(1/0x2319530) [ CSSD]2013-05-23 16:59:29.327 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:29.327 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319530 with con 0x22e5280, requested num -1 [ CSSD]2013-05-23 16:59:29.327 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:29.327 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 5 (0x7f431bfbf290) [ CSSD]2013-05-23 16:59:29.327 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf290 (RPC#5) tag(5002d) sent to node 1 [ CSSD]2013-05-23 16:59:29.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0005002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:29.327 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014938, gin 1, member 0 [ CSSD]2013-05-23 16:59:29.327 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:59:29.327 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:59:29.327 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:59:29.328 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf290 (RPC#5) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:29.328 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbf290, ret 0, client 0x2319530 member 0x22e4ef0 [ CSSD]2013-05-23 16:59:29.328 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319530 [ CSSD]2013-05-23 16:59:29.328 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 5 [ CSSD]2013-05-23 16:59:29.335 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:29.335 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319530), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:29.335 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:29.335 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 6 (0x7f431bfbf310) [ CSSD]2013-05-23 16:59:29.335 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf310 (RPC#6) tag(6002d) sent to node 1 [ CSSD]2013-05-23 16:59:29.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0006002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:29.335 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:29.335 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf310 (RPC#6) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:29.335 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbf310, ret 0, client 0x2319530 member 0x22e4ef0 [ CSSD]2013-05-23 16:59:29.336 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 6 [ CSSD]2013-05-23 16:59:29.336 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:59:29.538 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5000 [ CSSD]2013-05-23 16:59:29.538 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5000) con(0x22e36e0) skgpid ospid 6130 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:59:29.726 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:30.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:30.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:31.751 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:32.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:32.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:33.740 [1191196992] >TRACE: clssgmAllocProc: (0x22e4dc0) allocated [ CSSD]2013-05-23 16:59:33.741 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:59:33.742 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4dc0 - 1,2,3 [ CSSD]2013-05-23 16:59:33.742 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e5220) proc(0x22e4dc0) pid(6164) version 11:1:1:4 [ CSSD]2013-05-23 16:59:33.748 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4dc0), client(1/0x22e5460) [ CSSD]2013-05-23 16:59:33.748 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:59:33.748 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5460 with con 0x22e4bc0, requested num -1 [ CSSD]2013-05-23 16:59:33.748 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:33.748 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 7 (0x7f431bfbf390) [ CSSD]2013-05-23 16:59:33.748 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf390 (RPC#7) tag(7002d) sent to node 1 [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0007002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014939, gin 1, member 0 [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf390 (RPC#7) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbf390, ret 0, client 0x22e5460 member 0x22e5130 [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5460 [ CSSD]2013-05-23 16:59:33.749 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 7 [ CSSD]2013-05-23 16:59:33.756 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5460) [ CSSD]2013-05-23 16:59:33.756 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:33.756 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e5130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:33.756 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 8 (0x7f431bfbf410) [ CSSD]2013-05-23 16:59:33.756 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf410 (RPC#8) tag(8002d) sent to node 1 [ CSSD]2013-05-23 16:59:33.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0008002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:33.756 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:33.756 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf410 (RPC#8) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:33.756 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbf410, ret 0, client 0x22e5460 member 0x22e5130 [ CSSD]2013-05-23 16:59:33.756 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 8 [ CSSD]2013-05-23 16:59:33.756 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e5130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:59:33.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:33.939 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:33.939 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:33.959 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4dc0 [ CSSD]2013-05-23 16:59:33.959 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4dc0) con(0x22e5220) skgpid ospid 6164 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:59:34.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:34.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:35.831 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:36.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:36.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:37.141 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:59:37.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 9 (0x7f431bfbf490) [ CSSD]2013-05-23 16:59:37.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf490 (RPC#9) tag(9002d) sent to node 1 [ CSSD]2013-05-23 16:59:37.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:37.142 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 9 (0x7f431bfbf490) [ CSSD]2013-05-23 16:59:37.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf490 (RPC#9) state 6, flags 0x202 [ CSSD]2013-05-23 16:59:37.142 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf490, ret 0, client (nil) [ CSSD]2013-05-23 16:59:37.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 9 [ CSSD]2013-05-23 16:59:37.841 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:38.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:38.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:38.943 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:38.943 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:39.848 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:40.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:40.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:41.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:42.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:42.616 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:43.904 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:43.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:43.922 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014940, gin 1, member 0 [ CSSD]2013-05-23 16:59:43.922 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:59:43.922 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:59:43.930 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:43.930 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:59:43.930 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319020) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:59:43.946 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:43.946 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:44.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:44.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:45.910 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:46.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:46.676 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:47.167 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:59:47.168 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 10 (0x7f431bfbf510) [ CSSD]2013-05-23 16:59:47.168 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf510 (RPC#10) tag(a002d) sent to node 1 [ CSSD]2013-05-23 16:59:47.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:47.168 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 10 (0x7f431bfbf510) [ CSSD]2013-05-23 16:59:47.168 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf510 (RPC#10) state 6, flags 0x202 [ CSSD]2013-05-23 16:59:47.169 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf510, ret 0, client (nil) [ CSSD]2013-05-23 16:59:47.169 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 10 [ CSSD]2013-05-23 16:59:47.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:48.701 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:48.701 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:48.950 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:48.950 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:48.995 [1191196992] >TRACE: clssgmAllocProc: (0x22e50a0) allocated [ CSSD]2013-05-23 16:59:48.996 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 16:59:48.997 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e50a0 - 1,2,3 [ CSSD]2013-05-23 16:59:48.997 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e50a0) pid(6325) version 11:1:1:4 [ CSSD]2013-05-23 16:59:49.003 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e50a0), client(1/0x22e3ab0) [ CSSD]2013-05-23 16:59:49.003 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e3ab0) [ CSSD]2013-05-23 16:59:49.003 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e3ab0 with con 0x22e4ea0, requested num -1 [ CSSD]2013-05-23 16:59:49.003 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:49.003 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 11 (0x7f431bfbf590) [ CSSD]2013-05-23 16:59:49.003 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf590 (RPC#11) tag(b002d) sent to node 1 [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(000b002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014941, gin 1, member 0 [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf590 (RPC#11) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbf590, ret 0, client 0x22e3ab0 member 0x22e5410 [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e3ab0 [ CSSD]2013-05-23 16:59:49.004 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 11 [ CSSD]2013-05-23 16:59:49.010 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e3ab0) [ CSSD]2013-05-23 16:59:49.010 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e3ab0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:49.010 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e5410) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 16:59:49.010 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 12 (0x7f431bfbf610) [ CSSD]2013-05-23 16:59:49.011 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf610 (RPC#12) tag(c002d) sent to node 1 [ CSSD]2013-05-23 16:59:49.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(000c002d) incarnation(174631051) [ CSSD]2013-05-23 16:59:49.011 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 16:59:49.011 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf610 (RPC#12) state 6, flags 0x100 [ CSSD]2013-05-23 16:59:49.011 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbf610, ret 0, client 0x22e3ab0 member 0x22e5410 [ CSSD]2013-05-23 16:59:49.011 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 12 [ CSSD]2013-05-23 16:59:49.011 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e5410) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 16:59:49.214 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e50a0 [ CSSD]2013-05-23 16:59:49.214 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e50a0) con(0x22e36e0) skgpid ospid 6325 with 0 clients, refcount 0 [ CSSD]2013-05-23 16:59:49.928 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:50.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:50.711 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:51.934 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:52.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:52.717 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:52.953 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:52.953 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 16:59:53.964 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:54.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:54.770 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:55.971 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:56.795 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:56.795 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:57.103 [1191196992] >TRACE: clssgmAllocProc: (0x22e5080) allocated [ CSSD]2013-05-23 16:59:57.104 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 16:59:57.105 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5080 - 1,2,3 [ CSSD]2013-05-23 16:59:57.105 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e36e0) proc(0x22e5080) pid(5764/5764) version 11:1:1:4 [ CSSD]2013-05-23 16:59:57.111 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e5080), client(1/0x2319530) [ CSSD]2013-05-23 16:59:57.111 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2319530) [ CSSD]2013-05-23 16:59:57.111 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 16:59:57.111 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 16:59:57.191 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 16:59:57.192 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 13 (0x7f431bfbf690) [ CSSD]2013-05-23 16:59:57.192 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf690 (RPC#13) tag(d002d) sent to node 1 [ CSSD]2013-05-23 16:59:57.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:57.192 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 13 (0x7f431bfbf690) [ CSSD]2013-05-23 16:59:57.192 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf690 (RPC#13) state 6, flags 0x202 [ CSSD]2013-05-23 16:59:57.192 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf690, ret 0, client (nil) [ CSSD]2013-05-23 16:59:57.192 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 13 [ CSSD]2013-05-23 16:59:57.957 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 16:59:57.957 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 16:59:57.971 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 16:59:58.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:58.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 16:59:59.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:59.173 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014942, gin 1, member 0 [ CSSD]2013-05-23 16:59:59.173 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 16:59:59.173 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 16:59:59.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 16:59:59.181 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 16:59:59.181 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 16:59:59.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:00.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:00.813 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:02.005 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:02.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:02.819 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:02.960 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:02.960 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:04.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:04.251 [1191196992] >TRACE: clssgmAllocProc: (0x2322f90) allocated [ CSSD]2013-05-23 17:00:04.252 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:00:04.253 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322f90 - 1,2,3 [ CSSD]2013-05-23 17:00:04.253 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2322110) proc(0x2322f90) pid(6674) version 11:1:1:4 [ CSSD]2013-05-23 17:00:04.259 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322f90), client(1/0x22e8ec0) [ CSSD]2013-05-23 17:00:04.259 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8ec0) [ CSSD]2013-05-23 17:00:04.259 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8ec0 with con 0x22e89c0, requested num -1 [ CSSD]2013-05-23 17:00:04.259 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:04.259 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 14 (0x7f431bfbf710) [ CSSD]2013-05-23 17:00:04.260 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf710 (RPC#14) tag(e002d) sent to node 1 [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(000e002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014943, gin 1, member 0 [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9030/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf710 (RPC#14) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbf710, ret 0, client 0x22e8ec0 member 0x2322e80 [ CSSD]2013-05-23 17:00:04.260 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8ec0 [ CSSD]2013-05-23 17:00:04.261 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 14 [ CSSD]2013-05-23 17:00:04.267 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8ec0) [ CSSD]2013-05-23 17:00:04.267 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8ec0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:04.267 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2322e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:04.267 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 15 (0x7f431bfbf790) [ CSSD]2013-05-23 17:00:04.267 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf790 (RPC#15) tag(f002d) sent to node 1 [ CSSD]2013-05-23 17:00:04.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(000f002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:04.268 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:04.268 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf790 (RPC#15) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:04.268 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbf790, ret 0, client 0x22e8ec0 member 0x2322e80 [ CSSD]2013-05-23 17:00:04.268 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 15 [ CSSD]2013-05-23 17:00:04.268 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2322e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:00:04.470 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322f90 [ CSSD]2013-05-23 17:00:04.470 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322f90) con(0x2322110) skgpid ospid 6674 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:04.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:04.871 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:06.022 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:06.877 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:06.877 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:07.215 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:00:07.216 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 16 (0x7f431bfbf810) [ CSSD]2013-05-23 17:00:07.216 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf810 (RPC#16) tag(10002d) sent to node 1 [ CSSD]2013-05-23 17:00:07.216 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:07.216 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 16 (0x7f431bfbf810) [ CSSD]2013-05-23 17:00:07.216 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf810 (RPC#16) state 6, flags 0x202 [ CSSD]2013-05-23 17:00:07.216 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf810, ret 0, client (nil) [ CSSD]2013-05-23 17:00:07.216 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 16 [ CSSD]2013-05-23 17:00:07.964 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:07.964 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:08.005 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:00:08.006 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:00:08.006 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:00:08.006 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(6765) version 11:1:1:4 [ CSSD]2013-05-23 17:00:08.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:08.607 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:00:08.607 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 6765 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:08.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:08.882 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:10.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:10.892 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:10.892 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:12.045 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:12.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:12.918 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:12.968 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:12.968 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:14.056 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:14.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:14.424 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014944, gin 1, member 0 [ CSSD]2013-05-23 17:00:14.424 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:00:14.424 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:00:14.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:14.432 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:00:14.433 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:00:14.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:14.971 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:16.061 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:16.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:16.977 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:17.245 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:00:17.246 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 17 (0x7f431bfbf890) [ CSSD]2013-05-23 17:00:17.246 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf890 (RPC#17) tag(11002d) sent to node 1 [ CSSD]2013-05-23 17:00:17.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:17.246 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 17 (0x7f431bfbf890) [ CSSD]2013-05-23 17:00:17.246 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf890 (RPC#17) state 6, flags 0x202 [ CSSD]2013-05-23 17:00:17.246 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbf890, ret 0, client (nil) [ CSSD]2013-05-23 17:00:17.247 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 17 [ CSSD]2013-05-23 17:00:17.971 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:17.971 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:18.113 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:18.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:18.982 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:19.506 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:00:19.507 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:00:19.508 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:00:19.508 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(6939) version 11:1:1:4 [ CSSD]2013-05-23 17:00:19.514 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8d40) [ CSSD]2013-05-23 17:00:19.514 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:19.514 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8d40 with con 0x2322c70, requested num -1 [ CSSD]2013-05-23 17:00:19.514 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:19.514 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 18 (0x7f431bfbf910) [ CSSD]2013-05-23 17:00:19.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf910 (RPC#18) tag(12002d) sent to node 1 [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0012002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014945, gin 1, member 0 [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8eb0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf910 (RPC#18) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbf910, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8d40 [ CSSD]2013-05-23 17:00:19.515 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 18 [ CSSD]2013-05-23 17:00:19.522 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:19.522 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8d40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:19.522 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e91b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:19.522 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 19 (0x7f431bfbf990) [ CSSD]2013-05-23 17:00:19.522 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbf990 (RPC#19) tag(13002d) sent to node 1 [ CSSD]2013-05-23 17:00:19.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0013002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:19.522 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:19.522 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbf990 (RPC#19) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:19.522 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbf990, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:19.523 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 19 [ CSSD]2013-05-23 17:00:19.523 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e91b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:00:19.725 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:00:19.725 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 6939 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:20.119 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:20.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:20.993 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:22.125 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:22.975 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:22.975 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:23.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:23.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:24.136 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:25.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:25.072 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:25.765 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:25.765 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014946, gin 1, member 0 [ CSSD]2013-05-23 17:00:25.765 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:00:25.765 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:00:25.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:25.772 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:00:25.772 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:00:26.142 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:26.978 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:26.978 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:00:27.078 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:27.078 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:27.275 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:00:27.276 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 20 (0x7f431bfbfa10) [ CSSD]2013-05-23 17:00:27.276 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfa10 (RPC#20) tag(14002d) sent to node 1 [ CSSD]2013-05-23 17:00:27.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:27.276 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 20 (0x7f431bfbfa10) [ CSSD]2013-05-23 17:00:27.276 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfa10 (RPC#20) state 6, flags 0x202 [ CSSD]2013-05-23 17:00:27.276 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbfa10, ret 0, client (nil) [ CSSD]2013-05-23 17:00:27.277 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 20 [ CSSD]2013-05-23 17:00:28.153 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:29.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:29.083 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:29.583 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:00:29.584 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:00:29.585 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:00:29.585 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(7070) version 11:1:1:4 [ CSSD]2013-05-23 17:00:29.590 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8d40) [ CSSD]2013-05-23 17:00:29.591 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:29.591 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8d40 with con 0x2322c70, requested num -1 [ CSSD]2013-05-23 17:00:29.591 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:29.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 21 (0x7f431bfbfa90) [ CSSD]2013-05-23 17:00:29.591 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfa90 (RPC#21) tag(15002d) sent to node 1 [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0015002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014947, gin 1, member 0 [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8eb0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfa90 (RPC#21) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbfa90, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:29.591 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8d40 [ CSSD]2013-05-23 17:00:29.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 21 [ CSSD]2013-05-23 17:00:29.598 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:29.598 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8d40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:29.598 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e91b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:29.598 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 22 (0x7f431bfbfb10) [ CSSD]2013-05-23 17:00:29.598 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfb10 (RPC#22) tag(16002d) sent to node 1 [ CSSD]2013-05-23 17:00:29.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0016002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:29.599 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:29.599 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfb10 (RPC#22) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:29.599 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbfb10, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:29.599 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 22 [ CSSD]2013-05-23 17:00:29.599 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e91b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:00:29.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:29.671 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014948, gin 1, member 0 [ CSSD]2013-05-23 17:00:29.672 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:00:29.672 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:00:29.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:29.678 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:00:29.678 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:00:29.802 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:00:29.802 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 7070 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:30.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:31.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:31.094 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:31.981 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:31.982 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:32.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:33.003 [1191196992] >TRACE: clssgmAllocProc: (0x2322ff0) allocated [ CSSD]2013-05-23 17:00:33.005 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:00:33.005 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322ff0 - 1,2,3 [ CSSD]2013-05-23 17:00:33.005 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2322110) proc(0x2322ff0) pid(7116/7116) version 11:1:1:4 [ CSSD]2013-05-23 17:00:33.021 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322ff0), client(1/0x22e8c80) [ CSSD]2013-05-23 17:00:33.021 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e8c80) [ CSSD]2013-05-23 17:00:33.021 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:00:33.021 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:00:33.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:33.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:33.160 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322ff0), client(2/0x22e90b0) [ CSSD]2013-05-23 17:00:33.160 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22e90b0) [ CSSD]2013-05-23 17:00:33.160 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:00:33.160 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 17:00:34.216 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:34.765 [1191196992] >TRACE: clssgmAllocProc: (0x2327b30) allocated [ CSSD]2013-05-23 17:00:34.766 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:00:34.767 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327b30 - 1,2,3 [ CSSD]2013-05-23 17:00:34.767 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9860) proc(0x2327b30) pid(7130) version 11:1:1:4 [ CSSD]2013-05-23 17:00:34.773 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2327b30), client(1/0x23282a0) [ CSSD]2013-05-23 17:00:34.773 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23282a0) [ CSSD]2013-05-23 17:00:34.773 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23282a0 with con 0x2327d90, requested num -1 [ CSSD]2013-05-23 17:00:34.773 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:34.773 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 23 (0x7f431bfbfb90) [ CSSD]2013-05-23 17:00:34.773 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfb90 (RPC#23) tag(17002d) sent to node 1 [ CSSD]2013-05-23 17:00:34.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0017002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:34.773 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014949, gin 1, member 0 [ CSSD]2013-05-23 17:00:34.773 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328410/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:00:34.773 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:00:34.774 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:00:34.774 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfb90 (RPC#23) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:34.774 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbfb90, ret 0, client 0x23282a0 member 0x2327a10 [ CSSD]2013-05-23 17:00:34.774 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23282a0 [ CSSD]2013-05-23 17:00:34.774 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 23 [ CSSD]2013-05-23 17:00:34.779 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23282a0) [ CSSD]2013-05-23 17:00:34.779 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23282a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:34.779 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327a10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:34.779 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 24 (0x7f431bfbfc10) [ CSSD]2013-05-23 17:00:34.779 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfc10 (RPC#24) tag(18002d) sent to node 1 [ CSSD]2013-05-23 17:00:34.780 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0018002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:34.780 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:34.780 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfc10 (RPC#24) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:34.780 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbfc10, ret 0, client 0x23282a0 member 0x2327a10 [ CSSD]2013-05-23 17:00:34.780 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 24 [ CSSD]2013-05-23 17:00:34.780 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327a10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:00:34.982 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327b30 [ CSSD]2013-05-23 17:00:34.982 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327b30) con(0x22e9860) skgpid ospid 7130 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:35.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:35.172 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:36.222 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:36.984 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:36.985 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:37.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:37.177 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:37.301 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:00:37.301 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 25 (0x7f431bfbfc90) [ CSSD]2013-05-23 17:00:37.302 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfc90 (RPC#25) tag(19002d) sent to node 1 [ CSSD]2013-05-23 17:00:37.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:37.302 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 25 (0x7f431bfbfc90) [ CSSD]2013-05-23 17:00:37.302 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfc90 (RPC#25) state 6, flags 0x202 [ CSSD]2013-05-23 17:00:37.302 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbfc90, ret 0, client (nil) [ CSSD]2013-05-23 17:00:37.302 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 25 [ CSSD]2013-05-23 17:00:38.314 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:39.182 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:39.182 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:40.340 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:41.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:41.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:41.988 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:41.988 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:42.347 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:42.573 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22e90b0) [ CSSD]2013-05-23 17:00:42.573 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22e90b0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:00:42.573 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22e90b0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:00:42.573 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 17:00:42.578 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e8c80) [ CSSD]2013-05-23 17:00:42.578 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e8c80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:00:42.578 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e8c80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:00:42.578 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:00:42.578 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322ff0 [ CSSD]2013-05-23 17:00:42.578 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322ff0) con(0x2322110) skgpid 7116 ospid 7116 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:42.643 [1191196992] >TRACE: clssgmAllocProc: (0x22e75d0) allocated [ CSSD]2013-05-23 17:00:42.644 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:00:42.645 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e75d0 - 1,2,3 [ CSSD]2013-05-23 17:00:42.645 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9a50) proc(0x22e75d0) pid(7220/7220) version 11:1:1:4 [ CSSD]2013-05-23 17:00:42.660 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e75d0), client(1/0x23231b0) [ CSSD]2013-05-23 17:00:42.660 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23231b0) [ CSSD]2013-05-23 17:00:42.660 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:00:42.660 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:00:43.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:43.221 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:44.356 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:44.864 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23231b0) [ CSSD]2013-05-23 17:00:44.864 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23231b0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:00:44.864 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23231b0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:00:44.864 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:00:44.865 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e75d0 [ CSSD]2013-05-23 17:00:44.865 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e75d0) con(0x22e9a50) skgpid 7220 ospid 7220 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:44.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:44.922 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014950, gin 1, member 0 [ CSSD]2013-05-23 17:00:44.922 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:00:44.922 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:00:44.929 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:44.929 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:00:44.929 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:00:45.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:45.222 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:45.991 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:45.991 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:00:46.363 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:47.227 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:47.227 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:47.327 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:00:47.327 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 26 (0x7f431bfbfd10) [ CSSD]2013-05-23 17:00:47.327 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfd10 (RPC#26) tag(1a002d) sent to node 1 [ CSSD]2013-05-23 17:00:47.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:47.328 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 26 (0x7f431bfbfd10) [ CSSD]2013-05-23 17:00:47.328 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfd10 (RPC#26) state 6, flags 0x202 [ CSSD]2013-05-23 17:00:47.328 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbfd10, ret 0, client (nil) [ CSSD]2013-05-23 17:00:47.328 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 26 [ CSSD]2013-05-23 17:00:48.415 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:49.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:49.280 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:50.020 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:00:50.021 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:00:50.021 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:00:50.021 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(7278) version 11:1:1:4 [ CSSD]2013-05-23 17:00:50.027 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8d40) [ CSSD]2013-05-23 17:00:50.027 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:50.027 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8d40 with con 0x2322c70, requested num -1 [ CSSD]2013-05-23 17:00:50.028 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:50.028 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 27 (0x7f431bfbfd90) [ CSSD]2013-05-23 17:00:50.028 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfd90 (RPC#27) tag(1b002d) sent to node 1 [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(001b002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014951, gin 1, member 0 [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8eb0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfd90 (RPC#27) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbfd90, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8d40 [ CSSD]2013-05-23 17:00:50.028 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 27 [ CSSD]2013-05-23 17:00:50.035 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:50.035 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8d40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:50.035 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e91b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:50.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 28 (0x7f431bfbfe10) [ CSSD]2013-05-23 17:00:50.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfe10 (RPC#28) tag(1c002d) sent to node 1 [ CSSD]2013-05-23 17:00:50.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(001c002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:50.036 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:50.036 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfe10 (RPC#28) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:50.036 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbfe10, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:50.036 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 28 [ CSSD]2013-05-23 17:00:50.036 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e91b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:00:50.238 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:00:50.238 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 7278 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:50.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:50.995 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:50.995 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:51.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:51.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:52.428 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:53.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:53.292 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:54.437 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:55.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:55.304 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:55.999 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:00:55.999 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:00:56.443 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:57.310 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:57.310 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:00:57.354 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:00:57.355 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 29 (0x7f431bfbfe90) [ CSSD]2013-05-23 17:00:57.355 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbfe90 (RPC#29) tag(1d002d) sent to node 1 [ CSSD]2013-05-23 17:00:57.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:57.355 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 29 (0x7f431bfbfe90) [ CSSD]2013-05-23 17:00:57.355 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbfe90 (RPC#29) state 6, flags 0x202 [ CSSD]2013-05-23 17:00:57.355 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfbfe90, ret 0, client (nil) [ CSSD]2013-05-23 17:00:57.355 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 29 [ CSSD]2013-05-23 17:00:58.025 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:00:58.026 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:00:58.026 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:00:58.026 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(7382) version 11:1:1:4 [ CSSD]2013-05-23 17:00:58.033 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8d40) [ CSSD]2013-05-23 17:00:58.033 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:58.033 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8d40 with con 0x2322c70, requested num -1 [ CSSD]2013-05-23 17:00:58.033 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:58.033 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 30 (0x7f431bfbff10) [ CSSD]2013-05-23 17:00:58.033 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbff10 (RPC#30) tag(1e002d) sent to node 1 [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(001e002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014952, gin 1, member 0 [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8eb0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbff10 (RPC#30) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfbff10, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:58.033 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8d40 [ CSSD]2013-05-23 17:00:58.034 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 30 [ CSSD]2013-05-23 17:00:58.041 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:00:58.041 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8d40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:58.041 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e91b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:00:58.041 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 31 (0x7f431bfbff90) [ CSSD]2013-05-23 17:00:58.041 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfbff90 (RPC#31) tag(1f002d) sent to node 1 [ CSSD]2013-05-23 17:00:58.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(001f002d) incarnation(174631051) [ CSSD]2013-05-23 17:00:58.041 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:00:58.041 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfbff90 (RPC#31) state 6, flags 0x100 [ CSSD]2013-05-23 17:00:58.041 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfbff90, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:00:58.041 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 31 [ CSSD]2013-05-23 17:00:58.041 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e91b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:00:58.244 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:00:58.244 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 7382 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:00:58.455 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:00:59.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:00:59.320 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:00.182 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:00.182 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014953, gin 1, member 0 [ CSSD]2013-05-23 17:01:00.182 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:01:00.182 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:01:00.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:00.190 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:01:00.190 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:01:00.462 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:01.002 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:01.002 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:01.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:01.326 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:02.469 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:03.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:03.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:04.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:05.279 [1191196992] >TRACE: clssgmAllocProc: (0x2322f90) allocated [ CSSD]2013-05-23 17:01:05.280 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:01:05.281 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322f90 - 1,2,3 [ CSSD]2013-05-23 17:01:05.281 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2322110) proc(0x2322f90) pid(7447) version 11:1:1:4 [ CSSD]2013-05-23 17:01:05.287 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322f90), client(1/0x22e8ec0) [ CSSD]2013-05-23 17:01:05.287 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8ec0) [ CSSD]2013-05-23 17:01:05.287 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8ec0 with con 0x22e89c0, requested num -1 [ CSSD]2013-05-23 17:01:05.287 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:05.287 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 32 (0x7f431bfc0010) [ CSSD]2013-05-23 17:01:05.287 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0010 (RPC#32) tag(20002d) sent to node 1 [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0020002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014954, gin 1, member 0 [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e9030/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0010 (RPC#32) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0010, ret 0, client 0x22e8ec0 member 0x2322e80 [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8ec0 [ CSSD]2013-05-23 17:01:05.288 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 32 [ CSSD]2013-05-23 17:01:05.295 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8ec0) [ CSSD]2013-05-23 17:01:05.295 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8ec0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:05.295 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2322e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:05.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 33 (0x7f431bfc0090) [ CSSD]2013-05-23 17:01:05.295 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0090 (RPC#33) tag(21002d) sent to node 1 [ CSSD]2013-05-23 17:01:05.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0021002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:05.296 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:05.296 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0090 (RPC#33) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:05.296 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0090, ret 0, client 0x22e8ec0 member 0x2322e80 [ CSSD]2013-05-23 17:01:05.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 33 [ CSSD]2013-05-23 17:01:05.296 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2322e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:01:05.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:05.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:05.498 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322f90 [ CSSD]2013-05-23 17:01:05.498 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322f90) con(0x2322110) skgpid ospid 7447 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:06.006 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:06.006 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:06.526 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:07.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:07.390 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:07.394 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:01:07.395 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 34 (0x7f431bfc0110) [ CSSD]2013-05-23 17:01:07.395 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0110 (RPC#34) tag(22002d) sent to node 1 [ CSSD]2013-05-23 17:01:07.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:07.395 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 34 (0x7f431bfc0110) [ CSSD]2013-05-23 17:01:07.395 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0110 (RPC#34) state 6, flags 0x202 [ CSSD]2013-05-23 17:01:07.395 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0110, ret 0, client (nil) [ CSSD]2013-05-23 17:01:07.396 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 34 [ CSSD]2013-05-23 17:01:08.536 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:09.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:09.401 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:10.543 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:11.010 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:11.010 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:11.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:11.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:12.549 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:13.433 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:13.433 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:14.558 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:15.433 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:15.433 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014955, gin 1, member 0 [ CSSD]2013-05-23 17:01:15.433 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:01:15.433 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:01:15.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:15.440 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:01:15.440 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:01:15.484 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:15.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:16.014 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:16.014 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:16.565 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:17.419 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:01:17.419 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 35 (0x7f431bfc0190) [ CSSD]2013-05-23 17:01:17.419 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0190 (RPC#35) tag(23002d) sent to node 1 [ CSSD]2013-05-23 17:01:17.420 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:17.420 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 35 (0x7f431bfc0190) [ CSSD]2013-05-23 17:01:17.420 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0190 (RPC#35) state 6, flags 0x202 [ CSSD]2013-05-23 17:01:17.420 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0190, ret 0, client (nil) [ CSSD]2013-05-23 17:01:17.420 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 35 [ CSSD]2013-05-23 17:01:17.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:17.490 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:18.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:19.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:19.502 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:19.711 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:01:19.712 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:01:19.713 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:01:19.713 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(7601) version 11:1:1:4 [ CSSD]2013-05-23 17:01:19.719 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8d40) [ CSSD]2013-05-23 17:01:19.719 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:01:19.719 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22e8d40 with con 0x2322c70, requested num -1 [ CSSD]2013-05-23 17:01:19.719 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:01:19.719 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 36 (0x7f431bfc0210) [ CSSD]2013-05-23 17:01:19.719 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0210 (RPC#36) tag(24002d) sent to node 1 [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0024002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2014956, gin 1, member 0 [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8eb0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0210 (RPC#36) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0210, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8d40 [ CSSD]2013-05-23 17:01:19.720 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 36 [ CSSD]2013-05-23 17:01:19.726 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:01:19.726 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8d40), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:01:19.726 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e91b0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:01:19.726 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 37 (0x7f431bfc0290) [ CSSD]2013-05-23 17:01:19.726 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0290 (RPC#37) tag(25002d) sent to node 1 [ CSSD]2013-05-23 17:01:19.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0025002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:19.726 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:01:19.726 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0290 (RPC#37) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:19.726 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0290, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:01:19.727 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 37 [ CSSD]2013-05-23 17:01:19.727 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22e91b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:01:19.836 [1191196992] >TRACE: clssgmAllocProc: (0x2327090) allocated [ CSSD]2013-05-23 17:01:19.837 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:01:19.838 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327090 - 1,2,3 [ CSSD]2013-05-23 17:01:19.838 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23231a0) proc(0x2327090) pid(7606) version 11:1:1:4 [ CSSD]2013-05-23 17:01:19.852 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:01:19.854 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:01:19.856 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327090 [ CSSD]2013-05-23 17:01:19.856 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327090) con(0x23231a0) skgpid ospid 7606 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:19.929 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:01:19.929 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 7601 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:20.536 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:01:20.538 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:01:20.538 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:01:20.538 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(7622) version 11:1:1:4 [ CSSD]2013-05-23 17:01:20.545 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8d40) [ CSSD]2013-05-23 17:01:20.545 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:01:20.545 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e8d40 with con 0x2322c70, requested num -1 [ CSSD]2013-05-23 17:01:20.545 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:20.545 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 38 (0x7f431bfc0310) [ CSSD]2013-05-23 17:01:20.545 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0310 (RPC#38) tag(26002d) sent to node 1 [ CSSD]2013-05-23 17:01:20.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0026002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:20.545 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014957, gin 1, member 0 [ CSSD]2013-05-23 17:01:20.545 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e8eb0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:01:20.545 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:01:20.546 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:01:20.546 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0310 (RPC#38) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:20.546 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0310, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:01:20.546 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e8d40 [ CSSD]2013-05-23 17:01:20.546 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 38 [ CSSD]2013-05-23 17:01:20.552 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e8d40) [ CSSD]2013-05-23 17:01:20.552 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e8d40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:20.552 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e91b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:20.552 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 39 (0x7f431bfc0390) [ CSSD]2013-05-23 17:01:20.552 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0390 (RPC#39) tag(27002d) sent to node 1 [ CSSD]2013-05-23 17:01:20.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0027002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:20.552 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:20.552 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0390 (RPC#39) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:20.552 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0390, ret 0, client 0x22e8d40 member 0x22e91b0 [ CSSD]2013-05-23 17:01:20.553 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 39 [ CSSD]2013-05-23 17:01:20.553 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e91b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:01:20.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:20.755 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:01:20.755 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid ospid 7622 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:21.017 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:21.018 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:21.507 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:21.507 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:22.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:23.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:23.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:24.056 [1191196992] >TRACE: clssgmAllocProc: (0x2322e70) allocated [ CSSD]2013-05-23 17:01:24.057 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:01:24.057 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2322e70 - 1,2,3 [ CSSD]2013-05-23 17:01:24.057 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2321ff0) proc(0x2322e70) pid(7697/7697) version 11:1:1:4 [ CSSD]2013-05-23 17:01:24.073 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2322e70), client(1/0x22e8bc0) [ CSSD]2013-05-23 17:01:24.073 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e8bc0) [ CSSD]2013-05-23 17:01:24.074 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:01:24.074 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:01:24.088 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e8bc0) [ CSSD]2013-05-23 17:01:24.088 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e8bc0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:01:24.088 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e8bc0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:01:24.088 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:01:24.088 [1191196992] >TRACE: clssgmDeadProc: proc 0x2322e70 [ CSSD]2013-05-23 17:01:24.088 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2322e70) con(0x2321ff0) skgpid 7697 ospid 7697 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:24.398 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22d6ed0) [ CSSD]2013-05-23 17:01:24.398 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22d6ed0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:01:24.398 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22d6ed0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:01:24.398 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:01:24.405 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e6720) [ CSSD]2013-05-23 17:01:24.405 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e6720), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:01:24.405 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e6720), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:01:24.405 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:01:24.405 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:01:24.405 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2317510) skgpid 1923 ospid 1923 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:24.629 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:25.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:25.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:26.021 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:26.021 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:26.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:26.023 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014958, gin 1, member 0 [ CSSD]2013-05-23 17:01:26.023 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:01:26.023 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:01:26.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:26.030 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:01:26.030 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:01:26.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:27.444 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:01:27.444 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 40 (0x7f431bfc0410) [ CSSD]2013-05-23 17:01:27.445 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0410 (RPC#40) tag(28002d) sent to node 1 [ CSSD]2013-05-23 17:01:27.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:27.445 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 40 (0x7f431bfc0410) [ CSSD]2013-05-23 17:01:27.445 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0410 (RPC#40) state 6, flags 0x202 [ CSSD]2013-05-23 17:01:27.445 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0410, ret 0, client (nil) [ CSSD]2013-05-23 17:01:27.445 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 40 [ CSSD]2013-05-23 17:01:27.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:27.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:28.647 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:29.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:29.621 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:29.842 [1191196992] >TRACE: clssgmAllocProc: (0x2317510) allocated [ CSSD]2013-05-23 17:01:29.843 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:01:29.844 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317510 - 1,2,3 [ CSSD]2013-05-23 17:01:29.844 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f170) proc(0x2317510) pid(7755) version 11:1:1:4 [ CSSD]2013-05-23 17:01:29.850 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2317510), client(1/0x2321ad0) [ CSSD]2013-05-23 17:01:29.850 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:01:29.850 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321ad0 with con 0x22d6ed0, requested num -1 [ CSSD]2013-05-23 17:01:29.850 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:29.850 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 41 (0x7f431bfc0490) [ CSSD]2013-05-23 17:01:29.850 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0490 (RPC#41) tag(29002d) sent to node 1 [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0029002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014959, gin 1, member 0 [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0490 (RPC#41) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0490, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ad0 [ CSSD]2013-05-23 17:01:29.851 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 41 [ CSSD]2013-05-23 17:01:29.858 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:01:29.858 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321ad0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:29.858 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:29.858 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 42 (0x7f431bfc0510) [ CSSD]2013-05-23 17:01:29.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0510 (RPC#42) tag(2a002d) sent to node 1 [ CSSD]2013-05-23 17:01:29.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(002a002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:29.859 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:29.859 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0510 (RPC#42) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:29.859 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0510, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:01:29.859 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 42 [ CSSD]2013-05-23 17:01:29.859 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:01:30.061 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317510 [ CSSD]2013-05-23 17:01:30.061 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317510) con(0x230f170) skgpid ospid 7755 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:30.654 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:30.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:30.728 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014960, gin 1, member 0 [ CSSD]2013-05-23 17:01:30.728 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:01:30.728 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:01:30.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:30.736 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:01:30.736 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:01:31.025 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:31.025 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:31.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:31.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:32.660 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:33.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:33.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:34.670 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:35.028 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:35.028 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:01:35.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:35.686 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:35.797 [1191196992] >TRACE: clssgmAllocProc: (0x2317510) allocated [ CSSD]2013-05-23 17:01:35.798 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:01:35.798 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317510 - 1,2,3 [ CSSD]2013-05-23 17:01:35.798 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f170) proc(0x2317510) pid(7802) version 11:1:1:4 [ CSSD]2013-05-23 17:01:35.805 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2317510), client(1/0x2321ad0) [ CSSD]2013-05-23 17:01:35.805 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:01:35.805 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321ad0 with con 0x22d6ed0, requested num -1 [ CSSD]2013-05-23 17:01:35.805 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:35.805 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 43 (0x7f431bfc0590) [ CSSD]2013-05-23 17:01:35.805 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0590 (RPC#43) tag(2b002d) sent to node 1 [ CSSD]2013-05-23 17:01:35.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(002b002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:35.805 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014961, gin 1, member 0 [ CSSD]2013-05-23 17:01:35.805 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:01:35.806 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:01:35.806 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:01:35.806 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0590 (RPC#43) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:35.806 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0590, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:01:35.806 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ad0 [ CSSD]2013-05-23 17:01:35.806 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 43 [ CSSD]2013-05-23 17:01:35.812 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:01:35.813 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321ad0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:35.813 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:35.813 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 44 (0x7f431bfc0610) [ CSSD]2013-05-23 17:01:35.813 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0610 (RPC#44) tag(2c002d) sent to node 1 [ CSSD]2013-05-23 17:01:35.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(002c002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:35.813 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:35.813 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0610 (RPC#44) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:35.813 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0610, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:01:35.813 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 44 [ CSSD]2013-05-23 17:01:35.813 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:01:36.016 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317510 [ CSSD]2013-05-23 17:01:36.016 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317510) con(0x230f170) skgpid ospid 7802 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:36.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:37.477 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:01:37.477 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 45 (0x7f431bfc0690) [ CSSD]2013-05-23 17:01:37.477 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0690 (RPC#45) tag(2d002d) sent to node 1 [ CSSD]2013-05-23 17:01:37.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:37.478 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 45 (0x7f431bfc0690) [ CSSD]2013-05-23 17:01:37.478 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0690 (RPC#45) state 6, flags 0x202 [ CSSD]2013-05-23 17:01:37.478 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0690, ret 0, client (nil) [ CSSD]2013-05-23 17:01:37.478 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 45 [ CSSD]2013-05-23 17:01:37.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:37.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:38.728 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:39.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:39.723 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:40.032 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:40.032 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:40.734 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:41.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:41.729 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:42.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:43.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:43.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:44.831 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:45.036 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:45.036 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:45.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:45.786 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:45.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:45.979 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014962, gin 1, member 0 [ CSSD]2013-05-23 17:01:45.979 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:01:45.979 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:01:45.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:45.985 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:01:45.985 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:01:46.837 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:47.503 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:01:47.503 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 46 (0x7f431bfc0710) [ CSSD]2013-05-23 17:01:47.504 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0710 (RPC#46) tag(2e002d) sent to node 1 [ CSSD]2013-05-23 17:01:47.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:47.504 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 46 (0x7f431bfc0710) [ CSSD]2013-05-23 17:01:47.504 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0710 (RPC#46) state 6, flags 0x202 [ CSSD]2013-05-23 17:01:47.504 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0710, ret 0, client (nil) [ CSSD]2013-05-23 17:01:47.504 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 46 [ CSSD]2013-05-23 17:01:47.792 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:47.792 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:48.848 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:49.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:49.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:50.039 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:50.039 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:50.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:51.063 [1191196992] >TRACE: clssgmAllocProc: (0x205ecb0) allocated [ CSSD]2013-05-23 17:01:51.064 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:01:51.065 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205ecb0 - 1,2,3 [ CSSD]2013-05-23 17:01:51.065 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7cd0) proc(0x205ecb0) pid(7956) version 11:1:1:4 [ CSSD]2013-05-23 17:01:51.072 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205ecb0), client(1/0x22d6fa0) [ CSSD]2013-05-23 17:01:51.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d6fa0) [ CSSD]2013-05-23 17:01:51.072 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d6fa0 with con 0x22d74e0, requested num -1 [ CSSD]2013-05-23 17:01:51.072 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:51.072 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 47 (0x7f431bfc0790) [ CSSD]2013-05-23 17:01:51.072 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0790 (RPC#47) tag(2f002d) sent to node 1 [ CSSD]2013-05-23 17:01:51.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(002f002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:51.072 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014963, gin 1, member 0 [ CSSD]2013-05-23 17:01:51.072 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:01:51.073 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:01:51.073 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:01:51.073 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0790 (RPC#47) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:51.073 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0790, ret 0, client 0x22d6fa0 member 0x2302930 [ CSSD]2013-05-23 17:01:51.073 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d6fa0 [ CSSD]2013-05-23 17:01:51.073 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 47 [ CSSD]2013-05-23 17:01:51.079 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d6fa0) [ CSSD]2013-05-23 17:01:51.080 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d6fa0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:51.080 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:01:51.080 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 48 (0x7f431bfc0810) [ CSSD]2013-05-23 17:01:51.080 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0810 (RPC#48) tag(30002d) sent to node 1 [ CSSD]2013-05-23 17:01:51.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0030002d) incarnation(174631051) [ CSSD]2013-05-23 17:01:51.080 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:01:51.080 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0810 (RPC#48) state 6, flags 0x100 [ CSSD]2013-05-23 17:01:51.080 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0810, ret 0, client 0x22d6fa0 member 0x2302930 [ CSSD]2013-05-23 17:01:51.080 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 48 [ CSSD]2013-05-23 17:01:51.080 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:01:51.283 [1191196992] >TRACE: clssgmDeadProc: proc 0x205ecb0 [ CSSD]2013-05-23 17:01:51.283 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205ecb0) con(0x22d7cd0) skgpid ospid 7956 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:01:51.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:51.830 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:52.864 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:53.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:53.836 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:54.042 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:54.042 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:01:54.871 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:55.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:55.887 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:56.877 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:57.527 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:01:57.528 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 49 (0x7f431bfc0890) [ CSSD]2013-05-23 17:01:57.528 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0890 (RPC#49) tag(31002d) sent to node 1 [ CSSD]2013-05-23 17:01:57.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:57.528 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 49 (0x7f431bfc0890) [ CSSD]2013-05-23 17:01:57.528 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0890 (RPC#49) state 6, flags 0x202 [ CSSD]2013-05-23 17:01:57.528 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0890, ret 0, client (nil) [ CSSD]2013-05-23 17:01:57.528 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 49 [ CSSD]2013-05-23 17:01:57.892 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:57.892 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:01:58.929 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:01:59.046 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:01:59.046 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:01:59.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:01:59.904 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:00.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:01.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:01.232 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014964, gin 1, member 0 [ CSSD]2013-05-23 17:02:01.232 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:02:01.232 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:02:01.238 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:01.239 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:02:01.239 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:02:01.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:01.910 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:02.946 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:03.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:03.917 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:04.049 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:04.049 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:04.952 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:05.947 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:05.947 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:06.312 [1191196992] >TRACE: clssgmAllocProc: (0x2317510) allocated [ CSSD]2013-05-23 17:02:06.313 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:02:06.313 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317510 - 1,2,3 [ CSSD]2013-05-23 17:02:06.313 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7cd0) proc(0x2317510) pid(8109) version 11:1:1:4 [ CSSD]2013-05-23 17:02:06.329 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2317510), client(1/0x2321ad0) [ CSSD]2013-05-23 17:02:06.329 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:02:06.329 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321ad0 with con 0x22d6ed0, requested num -1 [ CSSD]2013-05-23 17:02:06.329 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:06.329 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 50 (0x7f431bfc0910) [ CSSD]2013-05-23 17:02:06.329 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0910 (RPC#50) tag(32002d) sent to node 1 [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0032002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014965, gin 1, member 0 [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0910 (RPC#50) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0910, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ad0 [ CSSD]2013-05-23 17:02:06.330 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 50 [ CSSD]2013-05-23 17:02:06.337 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:02:06.337 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321ad0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:06.337 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:06.337 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 51 (0x7f431bfc0990) [ CSSD]2013-05-23 17:02:06.337 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0990 (RPC#51) tag(33002d) sent to node 1 [ CSSD]2013-05-23 17:02:06.337 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0033002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:06.337 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:06.337 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0990 (RPC#51) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:06.337 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0990, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:02:06.337 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 51 [ CSSD]2013-05-23 17:02:06.337 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:06.540 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317510 [ CSSD]2013-05-23 17:02:06.540 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317510) con(0x22d7cd0) skgpid ospid 8109 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:06.958 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:07.551 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:02:07.552 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 52 (0x7f431bfc0a10) [ CSSD]2013-05-23 17:02:07.552 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0a10 (RPC#52) tag(34002d) sent to node 1 [ CSSD]2013-05-23 17:02:07.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:07.552 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 52 (0x7f431bfc0a10) [ CSSD]2013-05-23 17:02:07.552 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0a10 (RPC#52) state 6, flags 0x202 [ CSSD]2013-05-23 17:02:07.552 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0a10, ret 0, client (nil) [ CSSD]2013-05-23 17:02:07.553 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 52 [ CSSD]2013-05-23 17:02:07.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:07.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:08.052 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:08.052 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:02:08.072 [1191196992] >TRACE: clssgmAllocProc: (0x2317510) allocated [ CSSD]2013-05-23 17:02:08.073 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:02:08.074 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317510 - 1,2,3 [ CSSD]2013-05-23 17:02:08.074 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f170) proc(0x2317510) pid(8160) version 11:1:1:4 [ CSSD]2013-05-23 17:02:08.080 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2317510), client(1/0x2321ad0) [ CSSD]2013-05-23 17:02:08.080 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:02:08.080 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321ad0 with con 0x22d6ed0, requested num -1 [ CSSD]2013-05-23 17:02:08.080 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:08.080 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 53 (0x7f431bfc0a90) [ CSSD]2013-05-23 17:02:08.080 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0a90 (RPC#53) tag(35002d) sent to node 1 [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0035002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014966, gin 1, member 0 [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0a90 (RPC#53) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0a90, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ad0 [ CSSD]2013-05-23 17:02:08.081 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 53 [ CSSD]2013-05-23 17:02:08.087 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:02:08.087 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321ad0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:08.087 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:08.087 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 54 (0x7f431bfc0b10) [ CSSD]2013-05-23 17:02:08.088 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0b10 (RPC#54) tag(36002d) sent to node 1 [ CSSD]2013-05-23 17:02:08.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0036002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:08.088 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:08.088 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0b10 (RPC#54) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:08.088 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0b10, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:02:08.088 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 54 [ CSSD]2013-05-23 17:02:08.088 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:08.290 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317510 [ CSSD]2013-05-23 17:02:08.290 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317510) con(0x230f170) skgpid ospid 8160 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:09.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:09.954 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:09.954 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:11.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:11.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:11.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:13.055 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:13.055 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:13.066 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:13.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:13.984 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:15.072 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:16.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:16.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:16.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:16.482 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014967, gin 1, member 0 [ CSSD]2013-05-23 17:02:16.482 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:02:16.482 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:02:16.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:16.490 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:02:16.490 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:02:17.078 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:17.575 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:02:17.576 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 55 (0x7f431bfc0b90) [ CSSD]2013-05-23 17:02:17.576 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0b90 (RPC#55) tag(37002d) sent to node 1 [ CSSD]2013-05-23 17:02:17.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:17.576 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 55 (0x7f431bfc0b90) [ CSSD]2013-05-23 17:02:17.576 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0b90 (RPC#55) state 6, flags 0x202 [ CSSD]2013-05-23 17:02:17.576 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0b90, ret 0, client (nil) [ CSSD]2013-05-23 17:02:17.576 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 55 [ CSSD]2013-05-23 17:02:18.042 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:18.042 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:18.058 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:18.058 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:18.294 [1191196992] >TRACE: clssgmAllocProc: (0x2317510) allocated [ CSSD]2013-05-23 17:02:18.295 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:02:18.296 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317510 - 1,2,3 [ CSSD]2013-05-23 17:02:18.296 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f170) proc(0x2317510) pid(8452) version 11:1:1:4 [ CSSD]2013-05-23 17:02:18.435 [1191196992] >TRACE: clssgmAllocProc: (0x22e81f0) allocated [ CSSD]2013-05-23 17:02:18.437 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:02:18.437 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e81f0 - 1,2,3 [ CSSD]2013-05-23 17:02:18.437 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d75c0) proc(0x22e81f0) pid(8470) version 11:1:1:4 [ CSSD]2013-05-23 17:02:18.454 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:02:18.456 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:02:18.458 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e81f0 [ CSSD]2013-05-23 17:02:18.458 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e81f0) con(0x22d75c0) skgpid ospid 8470 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:18.528 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(75/0x22d6f90) [ CSSD]2013-05-23 17:02:18.528 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 75 (0x22d6f90) [ CSSD]2013-05-23 17:02:18.528 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22d6f90 with con 0x22d74e0, requested num -1 [ CSSD]2013-05-23 17:02:18.528 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:02:18.528 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 56 (0x7f431bfc0c10) [ CSSD]2013-05-23 17:02:18.528 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0c10 (RPC#56) tag(38002d) sent to node 1 [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0038002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2014968, gin 1, member 0 [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x2321ba0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0c10 (RPC#56) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0c10, ret 0, client 0x22d6f90 member 0x2302930 [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d6f90 [ CSSD]2013-05-23 17:02:18.529 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 56 [ CSSD]2013-05-23 17:02:18.537 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 75 (0x22d6f90) [ CSSD]2013-05-23 17:02:18.537 [1191196992] >TRACE: clssgmExitGrock: client 75 (0x22d6f90), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:02:18.537 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302930) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:02:18.537 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 57 (0x7f431bfc0c90) [ CSSD]2013-05-23 17:02:18.537 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0c90 (RPC#57) tag(39002d) sent to node 1 [ CSSD]2013-05-23 17:02:18.537 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0039002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:18.537 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:02:18.537 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0c90 (RPC#57) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:18.537 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0c90, ret 0, client 0x22d6f90 member 0x2302930 [ CSSD]2013-05-23 17:02:18.538 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 57 [ CSSD]2013-05-23 17:02:18.538 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x2302930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:18.539 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317510 [ CSSD]2013-05-23 17:02:18.539 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317510) con(0x230f170) skgpid ospid 8452 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:19.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:20.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:20.095 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:21.136 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:21.585 [1191196992] >TRACE: clssgmAllocProc: (0x205ecb0) allocated [ CSSD]2013-05-23 17:02:21.586 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:02:21.587 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205ecb0 - 1,2,3 [ CSSD]2013-05-23 17:02:21.587 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22fdb60) proc(0x205ecb0) pid(8504) version 11:1:1:4 [ CSSD]2013-05-23 17:02:21.593 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205ecb0), client(1/0x22d6fa0) [ CSSD]2013-05-23 17:02:21.593 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d6fa0) [ CSSD]2013-05-23 17:02:21.593 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d6fa0 with con 0x22d74e0, requested num -1 [ CSSD]2013-05-23 17:02:21.593 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:21.593 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 58 (0x7f431bfc0d10) [ CSSD]2013-05-23 17:02:21.593 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0d10 (RPC#58) tag(3a002d) sent to node 1 [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(003a002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014969, gin 1, member 0 [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0d10 (RPC#58) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0d10, ret 0, client 0x22d6fa0 member 0x2302930 [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d6fa0 [ CSSD]2013-05-23 17:02:21.594 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 58 [ CSSD]2013-05-23 17:02:21.602 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d6fa0) [ CSSD]2013-05-23 17:02:21.602 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d6fa0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:21.602 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:21.602 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 59 (0x7f431bfc0d90) [ CSSD]2013-05-23 17:02:21.602 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0d90 (RPC#59) tag(3b002d) sent to node 1 [ CSSD]2013-05-23 17:02:21.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(003b002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:21.602 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:21.602 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0d90 (RPC#59) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:21.602 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0d90, ret 0, client 0x22d6fa0 member 0x2302930 [ CSSD]2013-05-23 17:02:21.602 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 59 [ CSSD]2013-05-23 17:02:21.602 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:21.805 [1191196992] >TRACE: clssgmDeadProc: proc 0x205ecb0 [ CSSD]2013-05-23 17:02:21.805 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205ecb0) con(0x22fdb60) skgpid ospid 8504 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:22.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:22.101 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:23.062 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:23.062 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:23.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:24.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:24.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:25.153 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:26.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:26.136 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:26.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:26.273 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014970, gin 1, member 0 [ CSSD]2013-05-23 17:02:26.273 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:02:26.273 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:02:26.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:26.281 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:02:26.281 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:02:27.065 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:27.065 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:02:27.160 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:27.601 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:02:27.602 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 60 (0x7f431bfc0e10) [ CSSD]2013-05-23 17:02:27.602 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0e10 (RPC#60) tag(3c002d) sent to node 1 [ CSSD]2013-05-23 17:02:27.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:27.603 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 60 (0x7f431bfc0e10) [ CSSD]2013-05-23 17:02:27.603 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0e10 (RPC#60) state 6, flags 0x202 [ CSSD]2013-05-23 17:02:27.603 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc0e10, ret 0, client (nil) [ CSSD]2013-05-23 17:02:27.603 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 60 [ CSSD]2013-05-23 17:02:28.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:28.143 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:29.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:30.100 [1191196992] >TRACE: clssgmAllocProc: (0x205ecb0) allocated [ CSSD]2013-05-23 17:02:30.101 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:02:30.101 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205ecb0 - 1,2,3 [ CSSD]2013-05-23 17:02:30.101 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x205ecb0) pid(8671) version 11:1:1:4 [ CSSD]2013-05-23 17:02:30.107 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205ecb0), client(1/0x22d6fa0) [ CSSD]2013-05-23 17:02:30.107 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d6fa0) [ CSSD]2013-05-23 17:02:30.107 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d6fa0 with con 0x22d74e0, requested num -1 [ CSSD]2013-05-23 17:02:30.107 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:30.107 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 61 (0x7f431bfc0e90) [ CSSD]2013-05-23 17:02:30.108 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0e90 (RPC#61) tag(3d002d) sent to node 1 [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(003d002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014971, gin 1, member 0 [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321d20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0e90 (RPC#61) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0e90, ret 0, client 0x22d6fa0 member 0x2302930 [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d6fa0 [ CSSD]2013-05-23 17:02:30.108 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 61 [ CSSD]2013-05-23 17:02:30.115 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d6fa0) [ CSSD]2013-05-23 17:02:30.115 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d6fa0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:30.115 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:30.115 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 62 (0x7f431bfc0f10) [ CSSD]2013-05-23 17:02:30.115 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0f10 (RPC#62) tag(3e002d) sent to node 1 [ CSSD]2013-05-23 17:02:30.115 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(003e002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:30.115 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:30.115 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0f10 (RPC#62) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:30.115 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc0f10, ret 0, client 0x22d6fa0 member 0x2302930 [ CSSD]2013-05-23 17:02:30.115 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 62 [ CSSD]2013-05-23 17:02:30.115 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:30.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:30.196 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:30.318 [1191196992] >TRACE: clssgmDeadProc: proc 0x205ecb0 [ CSSD]2013-05-23 17:02:30.318 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205ecb0) con(0x231d6c0) skgpid ospid 8671 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:31.177 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:31.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:31.734 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014972, gin 1, member 0 [ CSSD]2013-05-23 17:02:31.735 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:02:31.735 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:02:31.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:31.743 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:02:31.743 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:02:32.069 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:32.069 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:32.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:32.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:33.227 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:34.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:34.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:35.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:36.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:36.320 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:36.840 [1191196992] >TRACE: clssgmAllocProc: (0x2317510) allocated [ CSSD]2013-05-23 17:02:36.841 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:02:36.842 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2317510 - 1,2,3 [ CSSD]2013-05-23 17:02:36.842 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x2317510) pid(8745) version 11:1:1:4 [ CSSD]2013-05-23 17:02:36.847 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2317510), client(1/0x2321ad0) [ CSSD]2013-05-23 17:02:36.847 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:02:36.847 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321ad0 with con 0x22d6ed0, requested num -1 [ CSSD]2013-05-23 17:02:36.847 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:36.847 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 63 (0x7f431bfc0f90) [ CSSD]2013-05-23 17:02:36.848 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc0f90 (RPC#63) tag(3f002d) sent to node 1 [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(003f002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014973, gin 1, member 0 [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321c40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc0f90 (RPC#63) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc0f90, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321ad0 [ CSSD]2013-05-23 17:02:36.848 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 63 [ CSSD]2013-05-23 17:02:36.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321ad0) [ CSSD]2013-05-23 17:02:36.854 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321ad0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:36.854 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:36.854 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 64 (0x7f431bfc1010) [ CSSD]2013-05-23 17:02:36.854 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1010 (RPC#64) tag(40002d) sent to node 1 [ CSSD]2013-05-23 17:02:36.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0040002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:36.855 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:36.855 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1010 (RPC#64) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:36.855 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1010, ret 0, client 0x2321ad0 member 0x2302950 [ CSSD]2013-05-23 17:02:36.855 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 64 [ CSSD]2013-05-23 17:02:36.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:37.057 [1191196992] >TRACE: clssgmDeadProc: proc 0x2317510 [ CSSD]2013-05-23 17:02:37.057 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2317510) con(0x231d6c0) skgpid ospid 8745 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:37.073 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:37.073 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:37.241 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:37.626 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:02:37.627 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 65 (0x7f431bfc1090) [ CSSD]2013-05-23 17:02:37.627 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1090 (RPC#65) tag(41002d) sent to node 1 [ CSSD]2013-05-23 17:02:37.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:37.627 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 65 (0x7f431bfc1090) [ CSSD]2013-05-23 17:02:37.627 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1090 (RPC#65) state 6, flags 0x202 [ CSSD]2013-05-23 17:02:37.627 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1090, ret 0, client (nil) [ CSSD]2013-05-23 17:02:37.628 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 65 [ CSSD]2013-05-23 17:02:38.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:38.326 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:39.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:40.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:40.336 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:41.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:42.077 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:42.077 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:42.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:42.341 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:43.258 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:44.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:44.394 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:45.265 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:46.080 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:46.080 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:02:46.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:46.419 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:46.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:46.988 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014974, gin 1, member 0 [ CSSD]2013-05-23 17:02:46.988 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:02:46.988 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:02:46.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:46.997 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:02:46.997 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:02:47.272 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:47.649 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:02:47.649 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 66 (0x7f431bfc1110) [ CSSD]2013-05-23 17:02:47.650 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1110 (RPC#66) tag(42002d) sent to node 1 [ CSSD]2013-05-23 17:02:47.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:47.650 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 66 (0x7f431bfc1110) [ CSSD]2013-05-23 17:02:47.650 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1110 (RPC#66) state 6, flags 0x202 [ CSSD]2013-05-23 17:02:47.650 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1110, ret 0, client (nil) [ CSSD]2013-05-23 17:02:47.650 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 66 [ CSSD]2013-05-23 17:02:48.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:48.465 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:49.279 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:49.828 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2302640) [ CSSD]2013-05-23 17:02:49.828 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2302640), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:02:49.828 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2302640), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:02:49.828 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 17:02:49.833 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:02:49.833 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:02:49.833 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:02:49.833 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:02:49.833 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302d20 [ CSSD]2013-05-23 17:02:49.833 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302d20) con(0x22f8800) skgpid 2960 ospid 2960 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:49.846 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22f37a0) [ CSSD]2013-05-23 17:02:49.846 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22f37a0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:02:49.846 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22f37a0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:02:49.846 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 17:02:49.853 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 17:02:49.853 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22d7880), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:02:49.853 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22d7880), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:02:49.853 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:02:49.853 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 17:02:49.853 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x2312940) skgpid 2954 ospid 2954 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:50.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:50.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:51.083 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:51.083 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:51.286 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:52.093 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:02:52.095 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:02:52.095 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:02:52.095 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(8961) version 11:1:1:4 [ CSSD]2013-05-23 17:02:52.102 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x22f3870) [ CSSD]2013-05-23 17:02:52.102 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3870) [ CSSD]2013-05-23 17:02:52.102 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3870 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 17:02:52.102 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:52.102 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 67 (0x7f431bfc1190) [ CSSD]2013-05-23 17:02:52.102 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1190 (RPC#67) tag(43002d) sent to node 1 [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0043002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014975, gin 1, member 0 [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1190 (RPC#67) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1190, ret 0, client 0x22f3870 member 0x22e01e0 [ CSSD]2013-05-23 17:02:52.102 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3870 [ CSSD]2013-05-23 17:02:52.103 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 67 [ CSSD]2013-05-23 17:02:52.109 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3870) [ CSSD]2013-05-23 17:02:52.109 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:52.109 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:02:52.109 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 68 (0x7f431bfc1210) [ CSSD]2013-05-23 17:02:52.109 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1210 (RPC#68) tag(44002d) sent to node 1 [ CSSD]2013-05-23 17:02:52.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0044002d) incarnation(174631051) [ CSSD]2013-05-23 17:02:52.110 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:02:52.110 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1210 (RPC#68) state 6, flags 0x100 [ CSSD]2013-05-23 17:02:52.110 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1210, ret 0, client 0x22f3870 member 0x22e01e0 [ CSSD]2013-05-23 17:02:52.110 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 68 [ CSSD]2013-05-23 17:02:52.110 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:02:52.312 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:02:52.312 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 8961 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:02:52.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:52.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:53.338 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:54.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:54.595 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:55.345 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:56.087 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:02:56.087 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:02:56.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:56.600 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:57.352 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:02:57.671 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:02:57.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 69 (0x7f431bfc1290) [ CSSD]2013-05-23 17:02:57.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1290 (RPC#69) tag(45002d) sent to node 1 [ CSSD]2013-05-23 17:02:57.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:57.672 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 69 (0x7f431bfc1290) [ CSSD]2013-05-23 17:02:57.672 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1290 (RPC#69) state 6, flags 0x202 [ CSSD]2013-05-23 17:02:57.672 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1290, ret 0, client (nil) [ CSSD]2013-05-23 17:02:57.673 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 69 [ CSSD]2013-05-23 17:02:58.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:02:58.606 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:02:59.361 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:00.090 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:00.090 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:03:00.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:00.618 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:01.369 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:02.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:02.240 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014976, gin 1, member 0 [ CSSD]2013-05-23 17:03:02.240 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:03:02.240 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:03:02.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:02.247 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:03:02.247 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:03:02.624 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:02.624 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:03.378 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:04.634 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:04.634 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:05.094 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:05.094 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:05.384 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:06.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:06.640 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:07.349 [1191196992] >TRACE: clssgmAllocProc: (0x231d6c0) allocated [ CSSD]2013-05-23 17:03:07.350 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:03:07.350 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231d6c0 - 1,2,3 [ CSSD]2013-05-23 17:03:07.350 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d20) proc(0x231d6c0) pid(9160) version 11:1:1:4 [ CSSD]2013-05-23 17:03:07.356 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231d6c0), client(1/0x22e2f80) [ CSSD]2013-05-23 17:03:07.356 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:03:07.356 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e2f80 with con 0x22f3860, requested num -1 [ CSSD]2013-05-23 17:03:07.356 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:07.356 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 70 (0x7f431bfc1310) [ CSSD]2013-05-23 17:03:07.356 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1310 (RPC#70) tag(46002d) sent to node 1 [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0046002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014977, gin 1, member 0 [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1310 (RPC#70) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1310, ret 0, client 0x22e2f80 member 0x22d7cd0 [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e2f80 [ CSSD]2013-05-23 17:03:07.357 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 70 [ CSSD]2013-05-23 17:03:07.363 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:03:07.363 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e2f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:07.363 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7cd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:07.363 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 71 (0x7f431bfc1390) [ CSSD]2013-05-23 17:03:07.363 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1390 (RPC#71) tag(47002d) sent to node 1 [ CSSD]2013-05-23 17:03:07.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0047002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:07.364 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:07.364 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1390 (RPC#71) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:07.364 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1390, ret 0, client 0x22e2f80 member 0x22d7cd0 [ CSSD]2013-05-23 17:03:07.364 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 71 [ CSSD]2013-05-23 17:03:07.364 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7cd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:03:07.391 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:07.566 [1191196992] >TRACE: clssgmDeadProc: proc 0x231d6c0 [ CSSD]2013-05-23 17:03:07.566 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231d6c0) con(0x2302d20) skgpid ospid 9160 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:03:07.697 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:03:07.698 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 72 (0x7f431bfc1410) [ CSSD]2013-05-23 17:03:07.698 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1410 (RPC#72) tag(48002d) sent to node 1 [ CSSD]2013-05-23 17:03:07.698 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:07.698 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 72 (0x7f431bfc1410) [ CSSD]2013-05-23 17:03:07.698 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1410 (RPC#72) state 6, flags 0x202 [ CSSD]2013-05-23 17:03:07.698 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1410, ret 0, client (nil) [ CSSD]2013-05-23 17:03:07.699 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 72 [ CSSD]2013-05-23 17:03:08.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:08.664 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:09.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:10.097 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:10.098 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:10.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:10.717 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:11.447 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:12.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:12.723 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:13.458 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:14.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:14.815 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:15.100 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:15.100 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:15.464 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:16.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:16.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:17.470 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:17.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:17.492 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014978, gin 1, member 0 [ CSSD]2013-05-23 17:03:17.492 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:03:17.492 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:03:17.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:17.499 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:03:17.499 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:03:17.723 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:03:17.724 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 73 (0x7f431bfc1490) [ CSSD]2013-05-23 17:03:17.724 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1490 (RPC#73) tag(49002d) sent to node 1 [ CSSD]2013-05-23 17:03:17.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:17.724 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 73 (0x7f431bfc1490) [ CSSD]2013-05-23 17:03:17.724 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1490 (RPC#73) state 6, flags 0x202 [ CSSD]2013-05-23 17:03:17.724 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1490, ret 0, client (nil) [ CSSD]2013-05-23 17:03:17.724 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 73 [ CSSD]2013-05-23 17:03:18.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:18.885 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:19.103 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:19.103 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:03:19.481 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:20.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:20.899 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:21.488 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:22.603 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:03:22.604 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:03:22.605 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:03:22.605 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(9317) version 11:1:1:4 [ CSSD]2013-05-23 17:03:22.621 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 17:03:22.621 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:03:22.621 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22f3860, requested num -1 [ CSSD]2013-05-23 17:03:22.621 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:22.621 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 74 (0x7f431bfc1510) [ CSSD]2013-05-23 17:03:22.621 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1510 (RPC#74) tag(4a002d) sent to node 1 [ CSSD]2013-05-23 17:03:22.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(004a002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014979, gin 1, member 0 [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1510 (RPC#74) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1510, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:03:22.622 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 74 [ CSSD]2013-05-23 17:03:22.629 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:03:22.629 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:22.629 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:22.629 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 75 (0x7f431bfc1590) [ CSSD]2013-05-23 17:03:22.629 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1590 (RPC#75) tag(4b002d) sent to node 1 [ CSSD]2013-05-23 17:03:22.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(004b002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:22.630 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:22.630 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1590 (RPC#75) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:22.630 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1590, ret 0, client 0x2312950 member 0x22e01e0 [ CSSD]2013-05-23 17:03:22.630 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 75 [ CSSD]2013-05-23 17:03:22.630 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:03:22.832 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:03:22.832 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 9317 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:03:22.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:22.905 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:23.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:24.106 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:24.107 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:24.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:24.916 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:25.546 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:26.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:26.532 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014980, gin 1, member 0 [ CSSD]2013-05-23 17:03:26.532 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:03:26.532 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:03:26.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:26.540 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:03:26.540 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:03:26.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:26.922 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:27.552 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:27.774 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:03:27.774 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 76 (0x7f431bfc1610) [ CSSD]2013-05-23 17:03:27.774 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1610 (RPC#76) tag(4c002d) sent to node 1 [ CSSD]2013-05-23 17:03:27.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:27.775 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 76 (0x7f431bfc1610) [ CSSD]2013-05-23 17:03:27.775 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1610 (RPC#76) state 6, flags 0x202 [ CSSD]2013-05-23 17:03:27.775 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1610, ret 0, client (nil) [ CSSD]2013-05-23 17:03:27.775 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 76 [ CSSD]2013-05-23 17:03:28.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:28.928 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:29.110 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:29.110 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:29.563 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:30.355 [1191196992] >TRACE: clssgmAllocProc: (0x231d6c0) allocated [ CSSD]2013-05-23 17:03:30.356 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:03:30.357 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231d6c0 - 1,2,3 [ CSSD]2013-05-23 17:03:30.357 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x231d6c0) pid(9422) version 11:1:1:4 [ CSSD]2013-05-23 17:03:30.363 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231d6c0), client(1/0x22f3870) [ CSSD]2013-05-23 17:03:30.363 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3870) [ CSSD]2013-05-23 17:03:30.363 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3870 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 17:03:30.363 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:30.363 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 77 (0x7f431bfc1690) [ CSSD]2013-05-23 17:03:30.363 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1690 (RPC#77) tag(4d002d) sent to node 1 [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(004d002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014981, gin 1, member 0 [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1690 (RPC#77) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1690, ret 0, client 0x22f3870 member 0x22e01e0 [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3870 [ CSSD]2013-05-23 17:03:30.364 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 77 [ CSSD]2013-05-23 17:03:30.370 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3870) [ CSSD]2013-05-23 17:03:30.370 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:30.370 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:30.370 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 78 (0x7f431bfc1710) [ CSSD]2013-05-23 17:03:30.370 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1710 (RPC#78) tag(4e002d) sent to node 1 [ CSSD]2013-05-23 17:03:30.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(004e002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:30.371 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:30.371 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1710 (RPC#78) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:30.371 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1710, ret 0, client 0x22f3870 member 0x22e01e0 [ CSSD]2013-05-23 17:03:30.371 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 78 [ CSSD]2013-05-23 17:03:30.371 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:03:30.573 [1191196992] >TRACE: clssgmDeadProc: proc 0x231d6c0 [ CSSD]2013-05-23 17:03:30.573 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231d6c0) con(0x205ecb0) skgpid ospid 9422 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:03:30.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:30.938 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:31.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:32.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:32.743 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014982, gin 1, member 0 [ CSSD]2013-05-23 17:03:32.743 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:03:32.743 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:03:32.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:32.750 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:03:32.750 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:03:32.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:32.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:33.580 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:34.114 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:34.114 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:34.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:34.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:35.587 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:36.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:36.971 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:37.593 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:37.803 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:03:37.803 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 79 (0x7f431bfc1790) [ CSSD]2013-05-23 17:03:37.803 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1790 (RPC#79) tag(4f002d) sent to node 1 [ CSSD]2013-05-23 17:03:37.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:37.804 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 79 (0x7f431bfc1790) [ CSSD]2013-05-23 17:03:37.804 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1790 (RPC#79) state 6, flags 0x202 [ CSSD]2013-05-23 17:03:37.804 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1790, ret 0, client (nil) [ CSSD]2013-05-23 17:03:37.804 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 79 [ CSSD]2013-05-23 17:03:37.871 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:03:37.872 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:03:37.873 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:03:37.873 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22dacd0) pid(9520) version 11:1:1:4 [ CSSD]2013-05-23 17:03:37.879 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x22e2f80) [ CSSD]2013-05-23 17:03:37.879 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:03:37.879 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e2f80 with con 0x22f3860, requested num -1 [ CSSD]2013-05-23 17:03:37.879 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:37.879 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 80 (0x7f431bfc1810) [ CSSD]2013-05-23 17:03:37.879 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1810 (RPC#80) tag(50002d) sent to node 1 [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0050002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014983, gin 1, member 0 [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1810 (RPC#80) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1810, ret 0, client 0x22e2f80 member 0x22fdb60 [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e2f80 [ CSSD]2013-05-23 17:03:37.880 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 80 [ CSSD]2013-05-23 17:03:37.886 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:03:37.886 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e2f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:37.886 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:37.886 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 81 (0x7f431bfc1890) [ CSSD]2013-05-23 17:03:37.886 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1890 (RPC#81) tag(51002d) sent to node 1 [ CSSD]2013-05-23 17:03:37.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0051002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:37.887 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:37.887 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1890 (RPC#81) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:37.887 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1890, ret 0, client 0x22e2f80 member 0x22fdb60 [ CSSD]2013-05-23 17:03:37.887 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 81 [ CSSD]2013-05-23 17:03:37.887 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:03:38.089 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:03:38.089 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2317510) skgpid ospid 9520 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:03:38.117 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:38.117 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:03:38.976 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:38.976 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:39.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:41.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:41.009 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:41.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:41.063 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2014984, gin 1, member 0 [ CSSD]2013-05-23 17:03:41.063 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:03:41.063 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:03:41.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:41.070 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:03:41.070 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:03:41.196 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:03:41.196 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:03:41.198 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:03:41.198 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:03:41.200 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:03:41.201 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:03:41.649 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:43.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:43.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:43.120 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:43.121 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:43.660 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:45.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:45.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:45.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:47.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:47.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:47.673 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:47.836 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:03:47.837 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 82 (0x7f431bfc1910) [ CSSD]2013-05-23 17:03:47.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1910 (RPC#82) tag(52002d) sent to node 1 [ CSSD]2013-05-23 17:03:47.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:47.837 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 82 (0x7f431bfc1910) [ CSSD]2013-05-23 17:03:47.837 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1910 (RPC#82) state 6, flags 0x202 [ CSSD]2013-05-23 17:03:47.837 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1910, ret 0, client (nil) [ CSSD]2013-05-23 17:03:47.837 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 82 [ CSSD]2013-05-23 17:03:48.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:48.004 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014985, gin 1, member 0 [ CSSD]2013-05-23 17:03:48.004 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:03:48.004 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:03:48.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:48.012 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:03:48.012 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:03:48.124 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:48.124 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:49.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:49.056 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:49.764 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:51.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:51.110 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:51.770 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:53.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:53.116 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:53.122 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 17:03:53.122 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:03:53.123 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 17:03:53.123 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d20) proc(0x22fc9c0) pid(9682) version 11:1:1:4 [ CSSD]2013-05-23 17:03:53.127 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:53.127 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:03:53.128 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22fc9c0), client(1/0x22e2f80) [ CSSD]2013-05-23 17:03:53.128 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:03:53.128 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e2f80 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 17:03:53.128 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:53.128 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 83 (0x7f431bfc1990) [ CSSD]2013-05-23 17:03:53.128 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1990 (RPC#83) tag(53002d) sent to node 1 [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0053002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014986, gin 1, member 0 [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1990 (RPC#83) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1990, ret 0, client 0x22e2f80 member 0x231d6c0 [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e2f80 [ CSSD]2013-05-23 17:03:53.128 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 83 [ CSSD]2013-05-23 17:03:53.134 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:03:53.134 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e2f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:53.134 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231d6c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:03:53.134 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 84 (0x7f431bfc1a10) [ CSSD]2013-05-23 17:03:53.134 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1a10 (RPC#84) tag(54002d) sent to node 1 [ CSSD]2013-05-23 17:03:53.134 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0054002d) incarnation(174631051) [ CSSD]2013-05-23 17:03:53.134 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:03:53.134 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1a10 (RPC#84) state 6, flags 0x100 [ CSSD]2013-05-23 17:03:53.134 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1a10, ret 0, client 0x22e2f80 member 0x231d6c0 [ CSSD]2013-05-23 17:03:53.134 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 84 [ CSSD]2013-05-23 17:03:53.134 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231d6c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:03:53.337 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 17:03:53.337 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x2302d20) skgpid ospid 9682 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:03:53.781 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:54.433 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ec760) [ CSSD]2013-05-23 17:03:54.433 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ec760), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:03:54.433 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ec760), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:03:54.433 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 27 [ CSSD]2013-05-23 17:03:54.433 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4f20 [ CSSD]2013-05-23 17:03:54.433 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4f20) con(0x22f3ea0) skgpid 3617 ospid 3617 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:03:55.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:55.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:55.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:57.130 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:03:57.130 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:03:57.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:57.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:57.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:03:57.870 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:03:57.871 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 85 (0x7f431bfc1a90) [ CSSD]2013-05-23 17:03:57.871 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1a90 (RPC#85) tag(55002d) sent to node 1 [ CSSD]2013-05-23 17:03:57.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:57.871 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 85 (0x7f431bfc1a90) [ CSSD]2013-05-23 17:03:57.871 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1a90 (RPC#85) state 6, flags 0x202 [ CSSD]2013-05-23 17:03:57.871 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1a90, ret 0, client (nil) [ CSSD]2013-05-23 17:03:57.871 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 85 [ CSSD]2013-05-23 17:03:59.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:03:59.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:03:59.794 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:01.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:01.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:01.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:02.133 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:02.134 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:03.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:03.257 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:03.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:03.266 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014987, gin 1, member 0 [ CSSD]2013-05-23 17:04:03.266 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:04:03.266 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:04:03.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:03.272 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:04:03.273 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:04:03.852 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:05.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:05.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:05.858 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:07.137 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:07.138 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:07.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:07.314 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:07.869 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:07.936 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:04:07.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 86 (0x7f431bfc1b10) [ CSSD]2013-05-23 17:04:07.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1b10 (RPC#86) tag(56002d) sent to node 1 [ CSSD]2013-05-23 17:04:07.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:07.937 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 86 (0x7f431bfc1b10) [ CSSD]2013-05-23 17:04:07.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1b10 (RPC#86) state 6, flags 0x202 [ CSSD]2013-05-23 17:04:07.937 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1b10, ret 0, client (nil) [ CSSD]2013-05-23 17:04:07.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 86 [ CSSD]2013-05-23 17:04:08.383 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 17:04:08.384 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:04:08.384 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 17:04:08.384 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22d7cd0) pid(9853) version 11:1:1:4 [ CSSD]2013-05-23 17:04:08.390 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x22d7cd0), client(1/0x22dada0) [ CSSD]2013-05-23 17:04:08.390 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 17:04:08.390 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dada0 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 17:04:08.390 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:08.390 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 87 (0x7f431bfc1b90) [ CSSD]2013-05-23 17:04:08.390 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1b90 (RPC#87) tag(57002d) sent to node 1 [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0057002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014988, gin 1, member 0 [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1b90 (RPC#87) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1b90, ret 0, client 0x22dada0 member 0x22e01e0 [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dada0 [ CSSD]2013-05-23 17:04:08.391 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 87 [ CSSD]2013-05-23 17:04:08.397 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dada0) [ CSSD]2013-05-23 17:04:08.397 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dada0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:08.397 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:08.397 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 88 (0x7f431bfc1c10) [ CSSD]2013-05-23 17:04:08.397 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1c10 (RPC#88) tag(58002d) sent to node 1 [ CSSD]2013-05-23 17:04:08.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0058002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:08.398 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:08.398 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1c10 (RPC#88) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:08.398 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1c10, ret 0, client 0x22dada0 member 0x22e01e0 [ CSSD]2013-05-23 17:04:08.398 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 88 [ CSSD]2013-05-23 17:04:08.398 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:04:08.601 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 17:04:08.601 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2317510) skgpid ospid 9853 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:04:09.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:09.326 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:09.875 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:11.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:11.352 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:11.882 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:12.141 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:12.141 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:13.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:13.357 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:13.892 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:15.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:15.408 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:15.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:17.145 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:17.145 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:17.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:17.413 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:17.949 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:17.961 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:04:17.961 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 89 (0x7f431bfc1c90) [ CSSD]2013-05-23 17:04:17.962 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1c90 (RPC#89) tag(59002d) sent to node 1 [ CSSD]2013-05-23 17:04:17.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:17.962 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 89 (0x7f431bfc1c90) [ CSSD]2013-05-23 17:04:17.962 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1c90 (RPC#89) state 6, flags 0x202 [ CSSD]2013-05-23 17:04:17.962 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1c90, ret 0, client (nil) [ CSSD]2013-05-23 17:04:17.962 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 89 [ CSSD]2013-05-23 17:04:18.214 [1191196992] >TRACE: clssgmAllocProc: (0x2302b40) allocated [ CSSD]2013-05-23 17:04:18.215 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:04:18.216 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302b40 - 1,2,3 [ CSSD]2013-05-23 17:04:18.216 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d20) proc(0x2302b40) pid(9741/9741) version 11:1:1:4 [ CSSD]2013-05-23 17:04:18.232 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x2302b40), client(1/0x22fdc30) [ CSSD]2013-05-23 17:04:18.232 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22fdc30) [ CSSD]2013-05-23 17:04:18.232 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:04:18.232 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 28 [ CSSD]2013-05-23 17:04:18.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:18.516 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014989, gin 1, member 0 [ CSSD]2013-05-23 17:04:18.516 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:04:18.516 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:04:18.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:18.523 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:04:18.523 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:04:19.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:19.426 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:19.955 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:21.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:21.432 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:21.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:22.148 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:22.149 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:23.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:23.437 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:23.633 [1191196992] >TRACE: clssgmAllocProc: (0x22d7750) allocated [ CSSD]2013-05-23 17:04:23.634 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:04:23.634 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7750 - 1,2,3 [ CSSD]2013-05-23 17:04:23.634 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7da0) proc(0x22d7750) pid(9996) version 11:1:1:4 [ CSSD]2013-05-23 17:04:23.640 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7750), client(1/0x22d7880) [ CSSD]2013-05-23 17:04:23.640 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 17:04:23.640 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7880 with con 0x22daa40, requested num -1 [ CSSD]2013-05-23 17:04:23.640 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:23.640 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 90 (0x7f431bfc1d10) [ CSSD]2013-05-23 17:04:23.640 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1d10 (RPC#90) tag(5a002d) sent to node 1 [ CSSD]2013-05-23 17:04:23.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(005a002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:23.640 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014990, gin 1, member 0 [ CSSD]2013-05-23 17:04:23.640 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:04:23.641 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:04:23.641 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:04:23.641 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1d10 (RPC#90) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:23.641 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1d10, ret 0, client 0x22d7880 member 0x205e170 [ CSSD]2013-05-23 17:04:23.641 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7880 [ CSSD]2013-05-23 17:04:23.641 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 90 [ CSSD]2013-05-23 17:04:23.647 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7880) [ CSSD]2013-05-23 17:04:23.647 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7880), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:23.647 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e170) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:23.647 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 91 (0x7f431bfc1d90) [ CSSD]2013-05-23 17:04:23.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1d90 (RPC#91) tag(5b002d) sent to node 1 [ CSSD]2013-05-23 17:04:23.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(005b002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:23.647 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:23.647 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1d90 (RPC#91) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:23.647 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1d90, ret 0, client 0x22d7880 member 0x205e170 [ CSSD]2013-05-23 17:04:23.647 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 91 [ CSSD]2013-05-23 17:04:23.647 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e170) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:04:23.850 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7750 [ CSSD]2013-05-23 17:04:23.850 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7750) con(0x22d7da0) skgpid ospid 9996 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:04:23.972 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:24.004 [1191196992] >TRACE: clssgmAllocProc: (0x22fc5a0) allocated [ CSSD]2013-05-23 17:04:24.005 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:04:24.005 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc5a0 - 1,2,3 [ CSSD]2013-05-23 17:04:24.005 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e02b0) proc(0x22fc5a0) pid(8942/8942) version 11:1:1:4 [ CSSD]2013-05-23 17:04:24.022 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22fc5a0), client(1/0x22dabd0) [ CSSD]2013-05-23 17:04:24.022 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22dabd0) [ CSSD]2013-05-23 17:04:24.022 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:04:24.022 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:04:25.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:25.449 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:25.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:26.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:26.794 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014991, gin 1, member 0 [ CSSD]2013-05-23 17:04:26.794 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:04:26.794 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:04:26.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:26.801 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:04:26.801 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:04:27.152 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:27.152 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:27.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:27.475 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:27.989 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:04:27.989 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:27.990 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 92 (0x7f431bfc1e10) [ CSSD]2013-05-23 17:04:27.990 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1e10 (RPC#92) tag(5c002d) sent to node 1 [ CSSD]2013-05-23 17:04:27.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:27.990 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 92 (0x7f431bfc1e10) [ CSSD]2013-05-23 17:04:27.990 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1e10 (RPC#92) state 6, flags 0x202 [ CSSD]2013-05-23 17:04:27.991 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1e10, ret 0, client (nil) [ CSSD]2013-05-23 17:04:27.991 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 92 [ CSSD]2013-05-23 17:04:28.079 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:04:28.079 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:04:28.080 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:04:28.081 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:04:28.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:28.162 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2014992, gin 1, member 0 [ CSSD]2013-05-23 17:04:28.162 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c0958d0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:04:28.162 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:04:28.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:28.169 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:04:28.169 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:04:29.507 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:29.507 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:29.996 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:30.626 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:04:30.628 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:04:30.628 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:04:30.628 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e230) proc(0x231ed30) pid(10085) version 11:1:1:4 [ CSSD]2013-05-23 17:04:30.635 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:04:30.635 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:04:30.635 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d7420, requested num -1 [ CSSD]2013-05-23 17:04:30.635 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:30.635 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 93 (0x7f431bfc1e90) [ CSSD]2013-05-23 17:04:30.635 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1e90 (RPC#93) tag(5d002d) sent to node 1 [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(005d002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014993, gin 1, member 0 [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4f20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1e90 (RPC#93) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc1e90, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:04:30.636 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 93 [ CSSD]2013-05-23 17:04:30.642 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:04:30.642 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:30.642 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231eb40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:30.642 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 94 (0x7f431bfc1f10) [ CSSD]2013-05-23 17:04:30.642 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1f10 (RPC#94) tag(5e002d) sent to node 1 [ CSSD]2013-05-23 17:04:30.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(005e002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:30.643 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:30.643 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1f10 (RPC#94) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:30.643 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc1f10, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 17:04:30.643 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 94 [ CSSD]2013-05-23 17:04:30.643 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231eb40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:04:30.845 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:04:30.845 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e230) skgpid ospid 10085 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:04:31.155 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:31.155 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:04:31.512 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:31.512 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:32.003 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:33.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:33.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:33.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:33.781 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014994, gin 1, member 0 [ CSSD]2013-05-23 17:04:33.781 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:04:33.781 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:04:33.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:33.789 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:04:33.789 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:04:34.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:35.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:35.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:36.060 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:36.158 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:36.159 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:37.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:37.536 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:38.020 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:04:38.020 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 95 (0x7f431bfc1f90) [ CSSD]2013-05-23 17:04:38.021 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc1f90 (RPC#95) tag(5f002d) sent to node 1 [ CSSD]2013-05-23 17:04:38.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:38.021 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 95 (0x7f431bfc1f90) [ CSSD]2013-05-23 17:04:38.021 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc1f90 (RPC#95) state 6, flags 0x202 [ CSSD]2013-05-23 17:04:38.021 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc1f90, ret 0, client (nil) [ CSSD]2013-05-23 17:04:38.021 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 95 [ CSSD]2013-05-23 17:04:38.070 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:38.892 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:04:38.893 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:04:38.894 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:04:38.894 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dfe10) proc(0x231ed30) pid(10156) version 11:1:1:4 [ CSSD]2013-05-23 17:04:38.900 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:04:38.900 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:04:38.900 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d7420, requested num -1 [ CSSD]2013-05-23 17:04:38.900 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:38.900 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 96 (0x7f431bfc2010) [ CSSD]2013-05-23 17:04:38.901 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2010 (RPC#96) tag(60002d) sent to node 1 [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0060002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014995, gin 1, member 0 [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4f20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2010 (RPC#96) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2010, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 17:04:38.901 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:04:38.902 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 96 [ CSSD]2013-05-23 17:04:38.908 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:04:38.908 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:38.908 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231eb40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:38.908 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 97 (0x7f431bfc2090) [ CSSD]2013-05-23 17:04:38.908 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2090 (RPC#97) tag(61002d) sent to node 1 [ CSSD]2013-05-23 17:04:38.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0061002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:38.908 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:38.909 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2090 (RPC#97) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:38.909 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2090, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 17:04:38.909 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 97 [ CSSD]2013-05-23 17:04:38.909 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231eb40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:04:39.111 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:04:39.111 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x22dfe10) skgpid ospid 10156 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:04:39.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:39.607 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:40.077 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:41.162 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:41.162 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:41.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:41.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:42.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:43.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:43.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:44.093 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:45.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:45.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:46.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:46.166 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:46.166 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:47.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:47.715 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:48.045 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:04:48.046 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 98 (0x7f431bfc2110) [ CSSD]2013-05-23 17:04:48.046 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2110 (RPC#98) tag(62002d) sent to node 1 [ CSSD]2013-05-23 17:04:48.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:48.046 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 98 (0x7f431bfc2110) [ CSSD]2013-05-23 17:04:48.046 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2110 (RPC#98) state 6, flags 0x202 [ CSSD]2013-05-23 17:04:48.046 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2110, ret 0, client (nil) [ CSSD]2013-05-23 17:04:48.046 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 98 [ CSSD]2013-05-23 17:04:48.150 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:49.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:49.053 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014996, gin 1, member 0 [ CSSD]2013-05-23 17:04:49.053 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:04:49.053 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:04:49.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:49.061 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:04:49.061 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:04:49.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:49.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:50.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:51.170 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:51.170 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:51.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:51.724 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:52.164 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:53.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:53.750 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:54.147 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:04:54.148 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:04:54.149 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:04:54.149 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f8800) proc(0x231ed30) pid(10311) version 11:1:1:4 [ CSSD]2013-05-23 17:04:54.155 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:04:54.155 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:04:54.155 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x205e3c0, requested num -1 [ CSSD]2013-05-23 17:04:54.155 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:54.155 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 99 (0x7f431bfc2190) [ CSSD]2013-05-23 17:04:54.155 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2190 (RPC#99) tag(63002d) sent to node 1 [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0063002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014997, gin 1, member 0 [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4f20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2190 (RPC#99) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2190, ret 0, client 0x22f1d30 member 0x22d75a0 [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:04:54.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 99 [ CSSD]2013-05-23 17:04:54.163 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:04:54.163 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:54.163 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d75a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:04:54.163 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 100 (0x7f431bfc2210) [ CSSD]2013-05-23 17:04:54.163 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2210 (RPC#100) tag(64002d) sent to node 1 [ CSSD]2013-05-23 17:04:54.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0064002d) incarnation(174631051) [ CSSD]2013-05-23 17:04:54.163 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:04:54.163 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2210 (RPC#100) state 6, flags 0x100 [ CSSD]2013-05-23 17:04:54.163 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2210, ret 0, client 0x22f1d30 member 0x22d75a0 [ CSSD]2013-05-23 17:04:54.164 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 100 [ CSSD]2013-05-23 17:04:54.164 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d75a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:04:54.174 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:54.366 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:04:54.366 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x22f8800) skgpid ospid 10311 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:04:55.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:55.761 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:56.173 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:04:56.174 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:04:56.181 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:57.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:57.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:04:58.124 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:04:58.125 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 101 (0x7f431bfc2290) [ CSSD]2013-05-23 17:04:58.125 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2290 (RPC#101) tag(65002d) sent to node 1 [ CSSD]2013-05-23 17:04:58.125 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:58.125 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 101 (0x7f431bfc2290) [ CSSD]2013-05-23 17:04:58.125 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2290 (RPC#101) state 6, flags 0x202 [ CSSD]2013-05-23 17:04:58.125 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2290, ret 0, client (nil) [ CSSD]2013-05-23 17:04:58.125 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 101 [ CSSD]2013-05-23 17:04:58.191 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:04:59.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:04:59.819 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:00.197 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:01.177 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:01.177 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:01.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:01.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:02.204 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:03.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:03.830 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:04.254 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:04.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:04.314 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2014998, gin 1, member 0 [ CSSD]2013-05-23 17:05:04.314 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:05:04.314 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:05:04.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:04.320 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:05:04.320 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:05:05.180 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:05.180 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:05:05.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:05.840 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:06.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:07.845 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:07.845 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:08.149 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:05:08.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 102 (0x7f431bfc2310) [ CSSD]2013-05-23 17:05:08.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2310 (RPC#102) tag(66002d) sent to node 1 [ CSSD]2013-05-23 17:05:08.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:08.150 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 102 (0x7f431bfc2310) [ CSSD]2013-05-23 17:05:08.150 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2310 (RPC#102) state 6, flags 0x202 [ CSSD]2013-05-23 17:05:08.150 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2310, ret 0, client (nil) [ CSSD]2013-05-23 17:05:08.150 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 102 [ CSSD]2013-05-23 17:05:08.271 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:08.471 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:05:08.472 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:05:08.472 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:05:08.472 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e230) proc(0x231ed30) pid(10463) version 11:1:1:4 [ CSSD]2013-05-23 17:05:09.161 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:05:09.161 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e230) skgpid ospid 10463 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:09.414 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:05:09.415 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:05:09.415 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:05:09.415 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e230) proc(0x231ed30) pid(10566) version 11:1:1:4 [ CSSD]2013-05-23 17:05:09.422 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:05:09.422 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:09.422 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d7420, requested num -1 [ CSSD]2013-05-23 17:05:09.422 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:09.422 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 103 (0x7f431bfc2390) [ CSSD]2013-05-23 17:05:09.422 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2390 (RPC#103) tag(67002d) sent to node 1 [ CSSD]2013-05-23 17:05:09.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0067002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:09.422 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2014999, gin 1, member 0 [ CSSD]2013-05-23 17:05:09.422 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4f20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:05:09.423 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:05:09.423 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:05:09.423 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2390 (RPC#103) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:09.423 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2390, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 17:05:09.423 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:05:09.423 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 103 [ CSSD]2013-05-23 17:05:09.429 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:09.429 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:09.429 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231eb40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:09.429 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 104 (0x7f431bfc2410) [ CSSD]2013-05-23 17:05:09.429 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2410 (RPC#104) tag(68002d) sent to node 1 [ CSSD]2013-05-23 17:05:09.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0068002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:09.429 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:09.429 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2410 (RPC#104) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:09.429 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2410, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 17:05:09.429 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 104 [ CSSD]2013-05-23 17:05:09.429 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231eb40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:05:09.632 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:05:09.632 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e230) skgpid ospid 10566 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:09.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:09.858 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:10.184 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:10.184 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:10.278 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:11.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:11.864 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:12.284 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:13.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:13.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:14.294 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:15.188 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:15.188 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:15.566 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x2302b40), client(2/0x205e360) [ CSSD]2013-05-23 17:05:15.566 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x205e360) [ CSSD]2013-05-23 17:05:15.566 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:05:15.566 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 17:05:15.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:15.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:16.301 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:17.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:17.946 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:18.175 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:05:18.175 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 105 (0x7f431bfc2490) [ CSSD]2013-05-23 17:05:18.175 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2490 (RPC#105) tag(69002d) sent to node 1 [ CSSD]2013-05-23 17:05:18.176 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:18.176 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 105 (0x7f431bfc2490) [ CSSD]2013-05-23 17:05:18.176 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2490 (RPC#105) state 6, flags 0x202 [ CSSD]2013-05-23 17:05:18.176 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2490, ret 0, client (nil) [ CSSD]2013-05-23 17:05:18.176 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 105 [ CSSD]2013-05-23 17:05:18.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:19.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:19.564 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015000, gin 1, member 0 [ CSSD]2013-05-23 17:05:19.564 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:05:19.564 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:05:19.571 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:19.571 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:05:19.571 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:05:19.881 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2319210) [ CSSD]2013-05-23 17:05:19.881 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2319210), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:05:19.881 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2319210), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:05:19.881 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 17:05:19.885 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2318d30) [ CSSD]2013-05-23 17:05:19.885 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2318d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:05:19.885 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2318d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:05:19.885 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:05:19.885 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318760 [ CSSD]2013-05-23 17:05:19.885 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318760) con(0x22ecd90) skgpid 4846 ospid 4846 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:20.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:20.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:20.191 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:20.191 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:20.308 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:22.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:22.063 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:22.315 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:24.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:24.069 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:24.194 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:24.194 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:05:24.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:24.667 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:05:24.668 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:05:24.669 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:05:24.669 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x231ed30) pid(10745) version 11:1:1:4 [ CSSD]2013-05-23 17:05:24.675 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:05:24.675 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:24.675 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:05:24.675 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:24.675 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 106 (0x7f431bfc2510) [ CSSD]2013-05-23 17:05:24.675 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2510 (RPC#106) tag(6a002d) sent to node 1 [ CSSD]2013-05-23 17:05:24.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(006a002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:24.675 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015001, gin 1, member 0 [ CSSD]2013-05-23 17:05:24.675 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6ed0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:05:24.676 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:05:24.676 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:05:24.676 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2510 (RPC#106) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:24.676 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2510, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:24.676 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:05:24.676 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 106 [ CSSD]2013-05-23 17:05:24.682 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:24.682 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:24.682 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d71d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:24.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 107 (0x7f431bfc2590) [ CSSD]2013-05-23 17:05:24.683 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2590 (RPC#107) tag(6b002d) sent to node 1 [ CSSD]2013-05-23 17:05:24.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(006b002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:24.683 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:24.683 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2590 (RPC#107) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:24.683 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2590, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:24.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 107 [ CSSD]2013-05-23 17:05:24.683 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d71d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:05:24.886 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:05:24.886 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e170) skgpid ospid 10745 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:26.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:26.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:26.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:27.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:27.055 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015002, gin 1, member 0 [ CSSD]2013-05-23 17:05:27.055 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:05:27.055 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:05:27.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:27.062 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:05:27.062 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:05:28.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:28.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:28.198 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:05:28.198 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 108 (0x7f431bfc2610) [ CSSD]2013-05-23 17:05:28.198 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2610 (RPC#108) tag(6c002d) sent to node 1 [ CSSD]2013-05-23 17:05:28.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:28.199 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 108 (0x7f431bfc2610) [ CSSD]2013-05-23 17:05:28.199 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2610 (RPC#108) state 6, flags 0x202 [ CSSD]2013-05-23 17:05:28.199 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2610, ret 0, client (nil) [ CSSD]2013-05-23 17:05:28.199 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 108 [ CSSD]2013-05-23 17:05:28.382 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:29.198 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:29.198 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:30.139 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:30.139 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:30.388 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:30.894 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:05:30.895 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:05:30.896 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:05:30.896 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x231ed30) pid(10826) version 11:1:1:4 [ CSSD]2013-05-23 17:05:30.902 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:05:30.902 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:30.902 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:05:30.902 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:30.902 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 109 (0x7f431bfc2690) [ CSSD]2013-05-23 17:05:30.902 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2690 (RPC#109) tag(6d002d) sent to node 1 [ CSSD]2013-05-23 17:05:30.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(006d002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:30.902 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015003, gin 1, member 0 [ CSSD]2013-05-23 17:05:30.902 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6ed0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:05:30.903 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:05:30.903 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:05:30.903 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2690 (RPC#109) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:30.903 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2690, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:30.903 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:05:30.903 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 109 [ CSSD]2013-05-23 17:05:30.909 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:30.909 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:30.909 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d71d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:30.909 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 110 (0x7f431bfc2710) [ CSSD]2013-05-23 17:05:30.909 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2710 (RPC#110) tag(6e002d) sent to node 1 [ CSSD]2013-05-23 17:05:30.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(006e002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:30.909 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:30.909 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2710 (RPC#110) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:30.909 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2710, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:30.910 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 110 [ CSSD]2013-05-23 17:05:30.910 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d71d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:05:31.112 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:05:31.112 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e170) skgpid ospid 10826 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:32.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:32.164 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:32.394 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:34.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:34.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:34.201 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:34.201 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:34.404 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:34.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:34.815 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015004, gin 1, member 0 [ CSSD]2013-05-23 17:05:34.815 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:05:34.815 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:05:34.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:34.823 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:05:34.823 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:05:36.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:36.242 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:36.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:38.204 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:38.204 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:05:38.225 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:05:38.225 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 111 (0x7f431bfc2790) [ CSSD]2013-05-23 17:05:38.226 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2790 (RPC#111) tag(6f002d) sent to node 1 [ CSSD]2013-05-23 17:05:38.226 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:38.226 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 111 (0x7f431bfc2790) [ CSSD]2013-05-23 17:05:38.226 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2790 (RPC#111) state 6, flags 0x202 [ CSSD]2013-05-23 17:05:38.226 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2790, ret 0, client (nil) [ CSSD]2013-05-23 17:05:38.226 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 111 [ CSSD]2013-05-23 17:05:38.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:38.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:38.462 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:39.919 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:05:39.921 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:05:39.921 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:05:39.921 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x231ed30) pid(10911) version 11:1:1:4 [ CSSD]2013-05-23 17:05:39.927 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:05:39.927 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:39.927 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:05:39.927 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:39.927 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 112 (0x7f431bfc2810) [ CSSD]2013-05-23 17:05:39.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2810 (RPC#112) tag(70002d) sent to node 1 [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0070002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015005, gin 1, member 0 [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6ed0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2810 (RPC#112) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2810, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:39.928 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:05:39.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 112 [ CSSD]2013-05-23 17:05:39.935 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:39.935 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:39.935 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d71d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:39.935 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 113 (0x7f431bfc2890) [ CSSD]2013-05-23 17:05:39.935 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2890 (RPC#113) tag(71002d) sent to node 1 [ CSSD]2013-05-23 17:05:39.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0071002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:39.935 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:39.935 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2890 (RPC#113) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:39.935 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2890, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:39.935 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 113 [ CSSD]2013-05-23 17:05:39.935 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d71d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:05:40.138 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:05:40.138 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e170) skgpid ospid 10911 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:40.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:40.260 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:40.468 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:42.265 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:42.265 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:42.475 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:43.207 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:43.208 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:44.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:44.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:44.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:46.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:46.324 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:46.492 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:48.211 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:48.211 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:48.251 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:05:48.251 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 114 (0x7f431bfc2910) [ CSSD]2013-05-23 17:05:48.251 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2910 (RPC#114) tag(72002d) sent to node 1 [ CSSD]2013-05-23 17:05:48.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:48.252 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 114 (0x7f431bfc2910) [ CSSD]2013-05-23 17:05:48.252 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2910 (RPC#114) state 6, flags 0x202 [ CSSD]2013-05-23 17:05:48.252 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2910, ret 0, client (nil) [ CSSD]2013-05-23 17:05:48.252 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 114 [ CSSD]2013-05-23 17:05:48.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:48.348 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:48.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:50.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:50.066 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015006, gin 1, member 0 [ CSSD]2013-05-23 17:05:50.066 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:05:50.066 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:05:50.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:50.073 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:05:50.073 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:05:50.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:50.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:50.511 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:52.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:52.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:52.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:53.215 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:53.216 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:05:54.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:54.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:54.567 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:55.175 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:05:55.177 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:05:55.177 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:05:55.177 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x231ed30) pid(11056) version 11:1:1:4 [ CSSD]2013-05-23 17:05:55.184 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:05:55.184 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:55.184 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:05:55.184 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:55.184 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 115 (0x7f431bfc2990) [ CSSD]2013-05-23 17:05:55.184 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2990 (RPC#115) tag(73002d) sent to node 1 [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0073002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015007, gin 1, member 0 [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6ed0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2990 (RPC#115) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2990, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:05:55.185 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 115 [ CSSD]2013-05-23 17:05:55.192 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:05:55.192 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:55.192 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d71d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:05:55.192 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 116 (0x7f431bfc2a10) [ CSSD]2013-05-23 17:05:55.192 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2a10 (RPC#116) tag(74002d) sent to node 1 [ CSSD]2013-05-23 17:05:55.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0074002d) incarnation(174631051) [ CSSD]2013-05-23 17:05:55.192 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:05:55.192 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2a10 (RPC#116) state 6, flags 0x100 [ CSSD]2013-05-23 17:05:55.192 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2a10, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:05:55.192 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 116 [ CSSD]2013-05-23 17:05:55.192 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d71d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:05:55.395 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:05:55.395 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e170) skgpid ospid 11056 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:05:56.524 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:56.524 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:56.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:05:57.218 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:05:57.219 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:05:58.276 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:05:58.277 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 117 (0x7f431bfc2a90) [ CSSD]2013-05-23 17:05:58.277 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2a90 (RPC#117) tag(75002d) sent to node 1 [ CSSD]2013-05-23 17:05:58.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:58.277 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 117 (0x7f431bfc2a90) [ CSSD]2013-05-23 17:05:58.277 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2a90 (RPC#117) state 6, flags 0x202 [ CSSD]2013-05-23 17:05:58.277 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2a90, ret 0, client (nil) [ CSSD]2013-05-23 17:05:58.277 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 117 [ CSSD]2013-05-23 17:05:58.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:05:58.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:05:58.584 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:00.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:00.541 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:00.591 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:02.222 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:02.223 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:02.547 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:02.547 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:02.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:04.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:04.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:04.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:05.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:05.316 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015008, gin 1, member 0 [ CSSD]2013-05-23 17:06:05.316 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:06:05.317 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:06:05.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:05.323 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:06:05.323 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:06:06.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:06.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:06.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:07.226 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:07.226 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:08.300 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:06:08.300 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 118 (0x7f431bfc2b10) [ CSSD]2013-05-23 17:06:08.301 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2b10 (RPC#118) tag(76002d) sent to node 1 [ CSSD]2013-05-23 17:06:08.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:08.301 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 118 (0x7f431bfc2b10) [ CSSD]2013-05-23 17:06:08.301 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2b10 (RPC#118) state 6, flags 0x202 [ CSSD]2013-05-23 17:06:08.301 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2b10, ret 0, client (nil) [ CSSD]2013-05-23 17:06:08.301 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 118 [ CSSD]2013-05-23 17:06:08.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:08.652 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:08.664 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:10.438 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:06:10.439 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:06:10.440 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:06:10.440 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e170) proc(0x231ed30) pid(11216) version 11:1:1:4 [ CSSD]2013-05-23 17:06:10.458 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:06:10.458 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:06:10.458 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:06:10.458 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:10.458 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 119 (0x7f431bfc2b90) [ CSSD]2013-05-23 17:06:10.458 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2b90 (RPC#119) tag(77002d) sent to node 1 [ CSSD]2013-05-23 17:06:10.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0077002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:10.458 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015009, gin 1, member 0 [ CSSD]2013-05-23 17:06:10.458 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6ed0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:06:10.459 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:06:10.459 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:06:10.459 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2b90 (RPC#119) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:10.459 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2b90, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:06:10.459 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:06:10.459 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 119 [ CSSD]2013-05-23 17:06:10.466 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:06:10.466 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:10.466 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d71d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:10.466 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 120 (0x7f431bfc2c10) [ CSSD]2013-05-23 17:06:10.466 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2c10 (RPC#120) tag(78002d) sent to node 1 [ CSSD]2013-05-23 17:06:10.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0078002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:10.466 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:10.466 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2c10 (RPC#120) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:10.466 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2c10, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:06:10.466 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 120 [ CSSD]2013-05-23 17:06:10.466 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d71d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:06:10.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:10.663 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:10.669 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:06:10.669 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x205e170) skgpid ospid 11216 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:10.671 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:11.229 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:11.229 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:06:12.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:12.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:12.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:14.687 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:14.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:14.730 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:16.233 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:16.233 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:16.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:16.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:16.736 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:18.324 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:06:18.324 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 121 (0x7f431bfc2c90) [ CSSD]2013-05-23 17:06:18.325 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2c90 (RPC#121) tag(79002d) sent to node 1 [ CSSD]2013-05-23 17:06:18.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:18.325 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 121 (0x7f431bfc2c90) [ CSSD]2013-05-23 17:06:18.325 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2c90 (RPC#121) state 6, flags 0x202 [ CSSD]2013-05-23 17:06:18.325 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2c90, ret 0, client (nil) [ CSSD]2013-05-23 17:06:18.325 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 121 [ CSSD]2013-05-23 17:06:18.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:18.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:18.761 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:20.004 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 17:06:20.005 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:06:20.005 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 17:06:20.005 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec640) proc(0x231ed30) pid(11320) version 11:1:1:4 [ CSSD]2013-05-23 17:06:20.012 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 17:06:20.012 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:06:20.012 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22f1d30 with con 0x22eee90, requested num -1 [ CSSD]2013-05-23 17:06:20.012 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:06:20.012 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 122 (0x7f431bfc2d10) [ CSSD]2013-05-23 17:06:20.012 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2d10 (RPC#122) tag(7a002d) sent to node 1 [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(007a002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015010, gin 1, member 0 [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6ed0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2d10 (RPC#122) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2d10, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:06:20.013 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 122 [ CSSD]2013-05-23 17:06:20.019 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:06:20.019 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:06:20.019 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d71d0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:06:20.019 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 123 (0x7f431bfc2d90) [ CSSD]2013-05-23 17:06:20.019 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2d90 (RPC#123) tag(7b002d) sent to node 1 [ CSSD]2013-05-23 17:06:20.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(007b002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:20.020 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:06:20.020 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2d90 (RPC#123) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:20.020 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2d90, ret 0, client 0x22f1d30 member 0x22d71d0 [ CSSD]2013-05-23 17:06:20.020 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 123 [ CSSD]2013-05-23 17:06:20.020 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22d71d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:06:20.137 [1191196992] >TRACE: clssgmAllocProc: (0x22f4c70) allocated [ CSSD]2013-05-23 17:06:20.137 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:06:20.138 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4c70 - 1,2,3 [ CSSD]2013-05-23 17:06:20.138 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7410) proc(0x22f4c70) pid(11334) version 11:1:1:4 [ CSSD]2013-05-23 17:06:20.153 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:06:20.154 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:06:20.157 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4c70 [ CSSD]2013-05-23 17:06:20.157 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4c70) con(0x22d7410) skgpid ospid 11334 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:20.222 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 17:06:20.222 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x22ec640) skgpid ospid 11320 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:20.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:20.587 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015011, gin 1, member 0 [ CSSD]2013-05-23 17:06:20.587 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:06:20.587 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:06:20.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:20.594 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:06:20.595 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:06:20.711 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:20.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:20.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:21.237 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:21.237 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:22.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:22.778 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:22.778 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:24.510 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2319530) [ CSSD]2013-05-23 17:06:24.510 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2319530), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:06:24.510 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2319530), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:06:24.510 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 27 [ CSSD]2013-05-23 17:06:24.510 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5080 [ CSSD]2013-05-23 17:06:24.510 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5080) con(0x22e36e0) skgpid 5764 ospid 5764 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:24.767 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:24.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:24.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:25.710 [1191196992] >TRACE: clssgmAllocProc: (0x231ed20) allocated [ CSSD]2013-05-23 17:06:25.711 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:06:25.711 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed20 - 1,2,3 [ CSSD]2013-05-23 17:06:25.711 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e240) proc(0x231ed20) pid(11386) version 11:1:1:4 [ CSSD]2013-05-23 17:06:25.717 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x231ed20), client(1/0x22dffb0) [ CSSD]2013-05-23 17:06:25.717 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dffb0) [ CSSD]2013-05-23 17:06:25.717 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dffb0 with con 0x231eb40, requested num -1 [ CSSD]2013-05-23 17:06:25.717 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:25.717 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 124 (0x7f431bfc2e10) [ CSSD]2013-05-23 17:06:25.717 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2e10 (RPC#124) tag(7c002d) sent to node 1 [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(007c002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015012, gin 1, member 0 [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e4e20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2e10 (RPC#124) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2e10, ret 0, client 0x22dffb0 member 0x22d7600 [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dffb0 [ CSSD]2013-05-23 17:06:25.718 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 124 [ CSSD]2013-05-23 17:06:25.724 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dffb0) [ CSSD]2013-05-23 17:06:25.724 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dffb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:25.724 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7600) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:25.724 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 125 (0x7f431bfc2e90) [ CSSD]2013-05-23 17:06:25.724 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2e90 (RPC#125) tag(7d002d) sent to node 1 [ CSSD]2013-05-23 17:06:25.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(007d002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:25.725 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:25.725 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2e90 (RPC#125) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:25.725 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc2e90, ret 0, client 0x22dffb0 member 0x22d7600 [ CSSD]2013-05-23 17:06:25.725 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 125 [ CSSD]2013-05-23 17:06:25.725 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:06:25.927 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed20 [ CSSD]2013-05-23 17:06:25.927 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed20) con(0x205e240) skgpid ospid 11386 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:26.240 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:26.240 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:26.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:26.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:26.837 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:27.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:27.318 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015013, gin 1, member 0 [ CSSD]2013-05-23 17:06:27.319 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:06:27.319 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:06:27.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:27.325 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:06:27.325 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:06:28.344 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:06:28.345 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 126 (0x7f431bfc2f10) [ CSSD]2013-05-23 17:06:28.345 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2f10 (RPC#126) tag(7e002d) sent to node 1 [ CSSD]2013-05-23 17:06:28.345 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:28.345 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 126 (0x7f431bfc2f10) [ CSSD]2013-05-23 17:06:28.345 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2f10 (RPC#126) state 6, flags 0x202 [ CSSD]2013-05-23 17:06:28.345 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc2f10, ret 0, client (nil) [ CSSD]2013-05-23 17:06:28.346 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 126 [ CSSD]2013-05-23 17:06:28.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:28.842 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:28.865 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:30.243 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:30.243 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:06:30.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:30.853 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:30.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:31.159 [1191196992] >TRACE: clssgmAllocProc: (0x231ed20) allocated [ CSSD]2013-05-23 17:06:31.161 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:06:31.161 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed20 - 1,2,3 [ CSSD]2013-05-23 17:06:31.161 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e5300) proc(0x231ed20) pid(11494) version 11:1:1:4 [ CSSD]2013-05-23 17:06:31.167 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x231ed20), client(1/0x22f1d30) [ CSSD]2013-05-23 17:06:31.167 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:06:31.167 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:06:31.167 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:31.167 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 127 (0x7f431bfc2f90) [ CSSD]2013-05-23 17:06:31.167 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc2f90 (RPC#127) tag(7f002d) sent to node 1 [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(007f002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015014, gin 1, member 0 [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc2f90 (RPC#127) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc2f90, ret 0, client 0x22f1d30 member 0x231ec10 [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:06:31.168 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 127 [ CSSD]2013-05-23 17:06:31.174 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:06:31.174 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:31.174 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231ec10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:31.174 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 128 (0x7f431bfc3010) [ CSSD]2013-05-23 17:06:31.174 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3010 (RPC#128) tag(80002d) sent to node 1 [ CSSD]2013-05-23 17:06:31.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0080002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:31.174 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:31.174 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3010 (RPC#128) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:31.174 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3010, ret 0, client 0x22f1d30 member 0x231ec10 [ CSSD]2013-05-23 17:06:31.174 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 128 [ CSSD]2013-05-23 17:06:31.174 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231ec10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:06:31.377 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed20 [ CSSD]2013-05-23 17:06:31.377 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed20) con(0x22e5300) skgpid ospid 11494 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:32.873 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:32.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:32.879 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:34.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:34.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:34.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:35.246 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:35.246 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:35.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:35.849 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015015, gin 1, member 0 [ CSSD]2013-05-23 17:06:35.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:06:35.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:06:35.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:35.856 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:06:35.856 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:06:36.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:36.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:36.936 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:38.385 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:06:38.385 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 129 (0x7f431bfc3090) [ CSSD]2013-05-23 17:06:38.385 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3090 (RPC#129) tag(81002d) sent to node 1 [ CSSD]2013-05-23 17:06:38.386 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:38.386 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 129 (0x7f431bfc3090) [ CSSD]2013-05-23 17:06:38.386 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3090 (RPC#129) state 6, flags 0x202 [ CSSD]2013-05-23 17:06:38.386 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3090, ret 0, client (nil) [ CSSD]2013-05-23 17:06:38.386 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 129 [ CSSD]2013-05-23 17:06:38.896 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:38.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:38.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:40.250 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:40.250 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:40.903 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:40.954 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:40.954 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:40.959 [1191196992] >TRACE: clssgmAllocProc: (0x231ed20) allocated [ CSSD]2013-05-23 17:06:40.960 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:06:40.961 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed20 - 1,2,3 [ CSSD]2013-05-23 17:06:40.961 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e240) proc(0x231ed20) pid(11581) version 11:1:1:4 [ CSSD]2013-05-23 17:06:40.977 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x231ed20), client(1/0x22dffb0) [ CSSD]2013-05-23 17:06:40.977 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dffb0) [ CSSD]2013-05-23 17:06:40.977 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dffb0 with con 0x22eee90, requested num -1 [ CSSD]2013-05-23 17:06:40.977 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:40.977 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 130 (0x7f431bfc3110) [ CSSD]2013-05-23 17:06:40.977 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3110 (RPC#130) tag(82002d) sent to node 1 [ CSSD]2013-05-23 17:06:40.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0082002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:40.977 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015016, gin 1, member 0 [ CSSD]2013-05-23 17:06:40.977 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:06:40.978 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:06:40.978 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:06:40.978 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3110 (RPC#130) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:40.978 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3110, ret 0, client 0x22dffb0 member 0x231ec10 [ CSSD]2013-05-23 17:06:40.978 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dffb0 [ CSSD]2013-05-23 17:06:40.978 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 130 [ CSSD]2013-05-23 17:06:40.985 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dffb0) [ CSSD]2013-05-23 17:06:40.985 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dffb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:40.985 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231ec10) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:40.985 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 131 (0x7f431bfc3190) [ CSSD]2013-05-23 17:06:40.985 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3190 (RPC#131) tag(83002d) sent to node 1 [ CSSD]2013-05-23 17:06:40.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0083002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:40.985 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:40.985 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3190 (RPC#131) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:40.985 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3190, ret 0, client 0x22dffb0 member 0x231ec10 [ CSSD]2013-05-23 17:06:40.985 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 131 [ CSSD]2013-05-23 17:06:40.985 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231ec10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:06:41.188 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed20 [ CSSD]2013-05-23 17:06:41.188 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed20) con(0x205e240) skgpid ospid 11581 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:42.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:42.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:42.993 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:45.000 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:45.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:45.032 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:45.254 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:45.254 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:46.444 [1191196992] >TRACE: clssgmAllocProc: (0x231ed20) allocated [ CSSD]2013-05-23 17:06:46.445 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:06:46.446 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed20 - 1,2,3 [ CSSD]2013-05-23 17:06:46.446 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e240) proc(0x231ed20) pid(10708/10708) version 11:1:1:4 [ CSSD]2013-05-23 17:06:46.462 [1191196992] >TRACE: clssgmRegisterClient: proc(56/0x231ed20), client(1/0x22dffb0) [ CSSD]2013-05-23 17:06:46.462 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22dffb0) [ CSSD]2013-05-23 17:06:46.462 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:06:46.462 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 28 [ CSSD]2013-05-23 17:06:47.007 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:47.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:47.038 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:48.415 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:06:48.415 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 132 (0x7f431bfc3210) [ CSSD]2013-05-23 17:06:48.416 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3210 (RPC#132) tag(84002d) sent to node 1 [ CSSD]2013-05-23 17:06:48.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:48.416 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 132 (0x7f431bfc3210) [ CSSD]2013-05-23 17:06:48.416 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3210 (RPC#132) state 6, flags 0x202 [ CSSD]2013-05-23 17:06:48.416 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3210, ret 0, client (nil) [ CSSD]2013-05-23 17:06:48.416 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 132 [ CSSD]2013-05-23 17:06:49.017 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:49.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:49.043 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:49.257 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:49.257 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:06:51.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:51.055 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:51.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:51.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:51.100 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015017, gin 1, member 0 [ CSSD]2013-05-23 17:06:51.100 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:06:51.100 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:06:51.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:51.140 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:06:51.140 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:06:53.093 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:53.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:53.100 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:54.260 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:54.260 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:06:55.098 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:55.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:55.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:56.225 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:06:56.226 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:06:56.227 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:06:56.227 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(11732) version 11:1:1:4 [ CSSD]2013-05-23 17:06:56.233 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:06:56.233 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:06:56.233 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:06:56.233 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:56.233 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 133 (0x7f431bfc3290) [ CSSD]2013-05-23 17:06:56.233 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3290 (RPC#133) tag(85002d) sent to node 1 [ CSSD]2013-05-23 17:06:56.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0085002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:56.233 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015018, gin 1, member 0 [ CSSD]2013-05-23 17:06:56.233 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:06:56.234 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:06:56.234 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:06:56.234 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3290 (RPC#133) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:56.234 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3290, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:06:56.234 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:06:56.234 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 133 [ CSSD]2013-05-23 17:06:56.241 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:06:56.241 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:56.241 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:06:56.241 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 134 (0x7f431bfc3310) [ CSSD]2013-05-23 17:06:56.241 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3310 (RPC#134) tag(86002d) sent to node 1 [ CSSD]2013-05-23 17:06:56.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0086002d) incarnation(174631051) [ CSSD]2013-05-23 17:06:56.242 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:06:56.242 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3310 (RPC#134) state 6, flags 0x100 [ CSSD]2013-05-23 17:06:56.242 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3310, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:06:56.242 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 134 [ CSSD]2013-05-23 17:06:56.242 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:06:56.444 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:06:56.444 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 11732 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:06:57.125 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:57.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:57.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:58.456 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:06:58.456 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 135 (0x7f431bfc3390) [ CSSD]2013-05-23 17:06:58.456 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3390 (RPC#135) tag(87002d) sent to node 1 [ CSSD]2013-05-23 17:06:58.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:58.457 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 135 (0x7f431bfc3390) [ CSSD]2013-05-23 17:06:58.457 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3390 (RPC#135) state 6, flags 0x202 [ CSSD]2013-05-23 17:06:58.457 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3390, ret 0, client (nil) [ CSSD]2013-05-23 17:06:58.457 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 135 [ CSSD]2013-05-23 17:06:59.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:06:59.163 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:06:59.177 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:06:59.264 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:06:59.264 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:01.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:01.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:01.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:03.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:03.181 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:03.195 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:03.267 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:03.267 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:07:05.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:05.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:05.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:06.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:06.365 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015019, gin 1, member 0 [ CSSD]2013-05-23 17:07:06.365 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:06.365 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:06.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:06.372 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:06.372 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:07.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:07.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:07.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:08.271 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:08.271 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:08.485 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:07:08.486 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 136 (0x7f431bfc3410) [ CSSD]2013-05-23 17:07:08.486 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3410 (RPC#136) tag(88002d) sent to node 1 [ CSSD]2013-05-23 17:07:08.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:08.487 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 136 (0x7f431bfc3410) [ CSSD]2013-05-23 17:07:08.487 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3410 (RPC#136) state 6, flags 0x202 [ CSSD]2013-05-23 17:07:08.487 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3410, ret 0, client (nil) [ CSSD]2013-05-23 17:07:08.487 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 136 [ CSSD]2013-05-23 17:07:09.217 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:09.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:09.262 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:11.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:11.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:11.276 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:11.492 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:07:11.493 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:07:11.494 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:07:11.494 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(11901) version 11:1:1:4 [ CSSD]2013-05-23 17:07:11.500 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:07:11.500 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:11.500 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:07:11.500 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:11.500 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 137 (0x7f431bfc3490) [ CSSD]2013-05-23 17:07:11.500 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3490 (RPC#137) tag(89002d) sent to node 1 [ CSSD]2013-05-23 17:07:11.500 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0089002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:11.500 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015020, gin 1, member 0 [ CSSD]2013-05-23 17:07:11.500 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:07:11.501 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:07:11.501 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:07:11.501 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3490 (RPC#137) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:11.501 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3490, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:11.501 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:07:11.501 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 137 [ CSSD]2013-05-23 17:07:11.507 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:11.507 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:11.507 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:11.507 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 138 (0x7f431bfc3510) [ CSSD]2013-05-23 17:07:11.507 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3510 (RPC#138) tag(8a002d) sent to node 1 [ CSSD]2013-05-23 17:07:11.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(008a002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:11.508 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:11.508 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3510 (RPC#138) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:11.508 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3510, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:11.508 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 138 [ CSSD]2013-05-23 17:07:11.508 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:07:11.710 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:07:11.710 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 11901 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:13.274 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:13.275 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:13.275 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:13.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:13.281 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:13.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:13.595 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015021, gin 1, member 0 [ CSSD]2013-05-23 17:07:13.595 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:13.595 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:13.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:13.603 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:13.603 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:15.321 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:15.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:15.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:17.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:17.339 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:17.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:18.277 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:18.277 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:18.515 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:07:18.516 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 139 (0x7f431bfc3590) [ CSSD]2013-05-23 17:07:18.516 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3590 (RPC#139) tag(8b002d) sent to node 1 [ CSSD]2013-05-23 17:07:18.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:18.517 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 139 (0x7f431bfc3590) [ CSSD]2013-05-23 17:07:18.517 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3590 (RPC#139) state 6, flags 0x202 [ CSSD]2013-05-23 17:07:18.517 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3590, ret 0, client (nil) [ CSSD]2013-05-23 17:07:18.517 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 139 [ CSSD]2013-05-23 17:07:18.617 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:07:18.618 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:07:18.619 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:07:18.619 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(12188) version 11:1:1:4 [ CSSD]2013-05-23 17:07:18.744 [1191196992] >TRACE: clssgmAllocProc: (0x22ecb60) allocated [ CSSD]2013-05-23 17:07:18.745 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:07:18.745 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ecb60 - 1,2,3 [ CSSD]2013-05-23 17:07:18.745 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2fa0) proc(0x22ecb60) pid(12198) version 11:1:1:4 [ CSSD]2013-05-23 17:07:18.762 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:07:18.764 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:07:18.766 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ecb60 [ CSSD]2013-05-23 17:07:18.766 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ecb60) con(0x22f2fa0) skgpid ospid 12198 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:18.840 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(76/0x22f1d30) [ CSSD]2013-05-23 17:07:18.840 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 76 (0x22f1d30) [ CSSD]2013-05-23 17:07:18.840 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22f1d30 with con 0x22f2fa0, requested num -1 [ CSSD]2013-05-23 17:07:18.840 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:07:18.840 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 140 (0x7f431bfc3610) [ CSSD]2013-05-23 17:07:18.840 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3610 (RPC#140) tag(8c002d) sent to node 1 [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(008c002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015022, gin 1, member 0 [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x22f2a10/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3610 (RPC#140) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3610, ret 0, client 0x22f1d30 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:07:18.841 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 140 [ CSSD]2013-05-23 17:07:18.848 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 76 (0x22f1d30) [ CSSD]2013-05-23 17:07:18.848 [1191196992] >TRACE: clssgmExitGrock: client 76 (0x22f1d30), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:07:18.848 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:07:18.848 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 141 (0x7f431bfc3690) [ CSSD]2013-05-23 17:07:18.848 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3690 (RPC#141) tag(8d002d) sent to node 1 [ CSSD]2013-05-23 17:07:18.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(008d002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:18.848 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:07:18.848 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3690 (RPC#141) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:18.848 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3690, ret 0, client 0x22f1d30 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:18.849 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 141 [ CSSD]2013-05-23 17:07:18.849 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:07:18.850 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:07:18.850 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 12188 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:19.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:19.351 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:19.378 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:21.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:21.356 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:21.385 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:21.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:21.627 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015023, gin 1, member 0 [ CSSD]2013-05-23 17:07:21.627 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:21.627 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:21.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:21.666 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:21.666 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:22.280 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:22.280 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:07:23.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:23.381 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:23.396 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:25.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:25.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:25.403 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:26.742 [1191196992] >TRACE: clssgmAllocProc: (0x22f2a10) allocated [ CSSD]2013-05-23 17:07:26.744 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:07:26.744 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2a10 - 1,2,3 [ CSSD]2013-05-23 17:07:26.744 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f37a0) proc(0x22f2a10) pid(12265) version 11:1:1:4 [ CSSD]2013-05-23 17:07:26.750 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f2a10), client(1/0x22f1d30) [ CSSD]2013-05-23 17:07:26.750 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:07:26.750 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22f3e00, requested num -1 [ CSSD]2013-05-23 17:07:26.750 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:26.750 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 142 (0x7f431bfc3710) [ CSSD]2013-05-23 17:07:26.751 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3710 (RPC#142) tag(8e002d) sent to node 1 [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(008e002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015024, gin 1, member 0 [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3710 (RPC#142) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3710, ret 0, client 0x22f1d30 member 0x22f2ff0 [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:07:26.751 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 142 [ CSSD]2013-05-23 17:07:26.758 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:07:26.758 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:26.758 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2ff0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:26.758 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 143 (0x7f431bfc3790) [ CSSD]2013-05-23 17:07:26.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3790 (RPC#143) tag(8f002d) sent to node 1 [ CSSD]2013-05-23 17:07:26.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(008f002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:26.759 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:26.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3790 (RPC#143) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:26.759 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3790, ret 0, client 0x22f1d30 member 0x22f2ff0 [ CSSD]2013-05-23 17:07:26.759 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 143 [ CSSD]2013-05-23 17:07:26.759 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2ff0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:07:26.961 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2a10 [ CSSD]2013-05-23 17:07:26.961 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2a10) con(0x22f37a0) skgpid ospid 12265 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:27.284 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:27.284 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:27.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:27.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:27.410 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:27.577 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:27.577 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015025, gin 1, member 0 [ CSSD]2013-05-23 17:07:27.577 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:27.577 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:27.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:27.616 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:27.616 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:28.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:07:28.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 144 (0x7f431bfc3810) [ CSSD]2013-05-23 17:07:28.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3810 (RPC#144) tag(90002d) sent to node 1 [ CSSD]2013-05-23 17:07:28.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:28.557 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 144 (0x7f431bfc3810) [ CSSD]2013-05-23 17:07:28.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3810 (RPC#144) state 6, flags 0x202 [ CSSD]2013-05-23 17:07:28.557 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3810, ret 0, client (nil) [ CSSD]2013-05-23 17:07:28.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 144 [ CSSD]2013-05-23 17:07:29.419 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:29.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:29.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:31.416 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:07:31.417 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:07:31.417 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:07:31.417 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(12335) version 11:1:1:4 [ CSSD]2013-05-23 17:07:31.424 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:07:31.424 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:31.424 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:07:31.424 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:31.424 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 145 (0x7f431bfc3890) [ CSSD]2013-05-23 17:07:31.424 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3890 (RPC#145) tag(91002d) sent to node 1 [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0091002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015026, gin 1, member 0 [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3890 (RPC#145) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3890, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:31.424 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:07:31.425 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 145 [ CSSD]2013-05-23 17:07:31.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:31.431 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:31.431 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:31.431 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:31.431 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 146 (0x7f431bfc3910) [ CSSD]2013-05-23 17:07:31.431 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3910 (RPC#146) tag(92002d) sent to node 1 [ CSSD]2013-05-23 17:07:31.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0092002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:31.431 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:31.431 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3910 (RPC#146) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:31.431 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3910, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:31.432 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 146 [ CSSD]2013-05-23 17:07:31.432 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:07:31.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:31.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:31.634 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:07:31.634 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 12335 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:32.287 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:32.288 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:33.476 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:33.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:33.513 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:35.483 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:35.544 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:35.544 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:36.877 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:36.877 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015027, gin 1, member 0 [ CSSD]2013-05-23 17:07:36.877 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:36.877 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:36.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:36.885 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:36.885 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:37.291 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:37.291 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:37.488 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:37.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:37.550 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:38.586 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:07:38.586 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 147 (0x7f431bfc3990) [ CSSD]2013-05-23 17:07:38.586 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3990 (RPC#147) tag(93002d) sent to node 1 [ CSSD]2013-05-23 17:07:38.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:38.587 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 147 (0x7f431bfc3990) [ CSSD]2013-05-23 17:07:38.587 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3990 (RPC#147) state 6, flags 0x202 [ CSSD]2013-05-23 17:07:38.587 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3990, ret 0, client (nil) [ CSSD]2013-05-23 17:07:38.587 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 147 [ CSSD]2013-05-23 17:07:39.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:39.562 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:39.562 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:41.294 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:41.294 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:07:41.506 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:41.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:41.587 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:41.999 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:07:42.000 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:07:42.001 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:07:42.001 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(12416) version 11:1:1:4 [ CSSD]2013-05-23 17:07:42.007 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:07:42.007 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:42.007 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:07:42.007 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:42.007 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 148 (0x7f431bfc3a10) [ CSSD]2013-05-23 17:07:42.007 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3a10 (RPC#148) tag(94002d) sent to node 1 [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0094002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015028, gin 1, member 0 [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3a10 (RPC#148) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3a10, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:07:42.008 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 148 [ CSSD]2013-05-23 17:07:42.017 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:42.017 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:42.017 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:42.017 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 149 (0x7f431bfc3a90) [ CSSD]2013-05-23 17:07:42.017 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3a90 (RPC#149) tag(95002d) sent to node 1 [ CSSD]2013-05-23 17:07:42.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0095002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:42.018 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:42.018 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3a90 (RPC#149) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:42.018 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3a90, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:42.018 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 149 [ CSSD]2013-05-23 17:07:42.018 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:07:42.221 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:07:42.221 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 12416 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:43.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:43.138 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015029, gin 1, member 0 [ CSSD]2013-05-23 17:07:43.138 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:43.138 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:43.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:43.145 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:43.145 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:43.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:43.593 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:43.593 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:45.524 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:45.644 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:45.644 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:46.298 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:46.298 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:47.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:47.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:47.650 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:48.606 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:07:48.606 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 150 (0x7f431bfc3b10) [ CSSD]2013-05-23 17:07:48.606 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3b10 (RPC#150) tag(96002d) sent to node 1 [ CSSD]2013-05-23 17:07:48.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:48.607 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 150 (0x7f431bfc3b10) [ CSSD]2013-05-23 17:07:48.607 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3b10 (RPC#150) state 6, flags 0x202 [ CSSD]2013-05-23 17:07:48.607 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3b10, ret 0, client (nil) [ CSSD]2013-05-23 17:07:48.607 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 150 [ CSSD]2013-05-23 17:07:49.580 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:49.662 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:49.662 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:51.302 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:51.302 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:51.586 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:51.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:51.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:52.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:52.128 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015030, gin 1, member 0 [ CSSD]2013-05-23 17:07:52.128 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:07:52.128 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:07:52.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:52.135 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:07:52.135 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:07:53.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:53.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:53.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:55.604 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:55.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:55.683 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:56.305 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:07:56.306 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:07:57.268 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:07:57.269 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:07:57.269 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:07:57.269 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(12649) version 11:1:1:4 [ CSSD]2013-05-23 17:07:57.276 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:07:57.276 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:57.276 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:07:57.276 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:57.276 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 151 (0x7f431bfc3b90) [ CSSD]2013-05-23 17:07:57.276 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3b90 (RPC#151) tag(97002d) sent to node 1 [ CSSD]2013-05-23 17:07:57.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0097002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:57.276 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015031, gin 1, member 0 [ CSSD]2013-05-23 17:07:57.276 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:07:57.277 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:07:57.277 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:07:57.277 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3b90 (RPC#151) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:57.277 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3b90, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:57.277 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:07:57.277 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 151 [ CSSD]2013-05-23 17:07:57.284 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:07:57.284 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:57.284 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:07:57.284 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 152 (0x7f431bfc3c10) [ CSSD]2013-05-23 17:07:57.284 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3c10 (RPC#152) tag(98002d) sent to node 1 [ CSSD]2013-05-23 17:07:57.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0098002d) incarnation(174631051) [ CSSD]2013-05-23 17:07:57.284 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:07:57.284 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3c10 (RPC#152) state 6, flags 0x100 [ CSSD]2013-05-23 17:07:57.284 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3c10, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:07:57.284 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 152 [ CSSD]2013-05-23 17:07:57.284 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:07:57.487 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:07:57.487 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 12649 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:07:57.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:57.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:57.689 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:07:58.626 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:07:58.626 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 153 (0x7f431bfc3c90) [ CSSD]2013-05-23 17:07:58.626 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3c90 (RPC#153) tag(99002d) sent to node 1 [ CSSD]2013-05-23 17:07:58.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:58.627 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 153 (0x7f431bfc3c90) [ CSSD]2013-05-23 17:07:58.627 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3c90 (RPC#153) state 6, flags 0x202 [ CSSD]2013-05-23 17:07:58.627 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3c90, ret 0, client (nil) [ CSSD]2013-05-23 17:07:58.627 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 153 [ CSSD]2013-05-23 17:07:59.631 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:07:59.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:07:59.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:01.309 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:01.309 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:01.638 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:01.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:01.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:03.688 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:03.753 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:03.754 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:05.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:05.765 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:05.765 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:06.313 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:06.313 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:07.379 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:07.379 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015032, gin 1, member 0 [ CSSD]2013-05-23 17:08:07.379 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:08:07.379 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:08:07.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:07.385 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:08:07.385 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:08:07.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:07.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:07.789 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:08.656 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:08:08.656 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 154 (0x7f431bfc3d10) [ CSSD]2013-05-23 17:08:08.656 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3d10 (RPC#154) tag(9a002d) sent to node 1 [ CSSD]2013-05-23 17:08:08.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:08.657 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 154 (0x7f431bfc3d10) [ CSSD]2013-05-23 17:08:08.657 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3d10 (RPC#154) state 6, flags 0x202 [ CSSD]2013-05-23 17:08:08.657 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3d10, ret 0, client (nil) [ CSSD]2013-05-23 17:08:08.657 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 154 [ CSSD]2013-05-23 17:08:09.711 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:09.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:09.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:11.317 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:11.317 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:11.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:11.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:11.868 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:12.524 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:08:12.525 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:08:12.526 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:08:12.526 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(12814) version 11:1:1:4 [ CSSD]2013-05-23 17:08:12.533 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:08:12.533 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:12.533 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:08:12.533 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:12.533 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 155 (0x7f431bfc3d90) [ CSSD]2013-05-23 17:08:12.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3d90 (RPC#155) tag(9b002d) sent to node 1 [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(009b002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015033, gin 1, member 0 [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3d90 (RPC#155) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3d90, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:08:12.534 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 155 [ CSSD]2013-05-23 17:08:12.541 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:12.541 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:12.541 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:12.541 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 156 (0x7f431bfc3e10) [ CSSD]2013-05-23 17:08:12.541 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3e10 (RPC#156) tag(9c002d) sent to node 1 [ CSSD]2013-05-23 17:08:12.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(009c002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:12.541 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:12.541 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3e10 (RPC#156) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:12.541 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3e10, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:12.542 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 156 [ CSSD]2013-05-23 17:08:12.542 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:08:12.744 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:08:12.744 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 12814 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:08:13.809 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:13.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:13.873 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:15.320 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:15.320 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:08:15.815 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:15.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:15.884 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:17.822 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:17.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:17.890 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:18.687 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:08:18.688 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 157 (0x7f431bfc3e90) [ CSSD]2013-05-23 17:08:18.688 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3e90 (RPC#157) tag(9d002d) sent to node 1 [ CSSD]2013-05-23 17:08:18.688 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:18.688 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 157 (0x7f431bfc3e90) [ CSSD]2013-05-23 17:08:18.688 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3e90 (RPC#157) state 6, flags 0x202 [ CSSD]2013-05-23 17:08:18.688 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc3e90, ret 0, client (nil) [ CSSD]2013-05-23 17:08:18.688 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 157 [ CSSD]2013-05-23 17:08:19.832 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:19.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:19.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:20.323 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:20.323 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:21.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:21.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:21.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:22.639 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:22.639 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015034, gin 1, member 0 [ CSSD]2013-05-23 17:08:22.639 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:08:22.639 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:08:22.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:22.646 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:08:22.646 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:08:23.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:23.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:23.995 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:25.327 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:25.327 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:25.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:26.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:26.047 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:27.791 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:08:27.792 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:08:27.793 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:08:27.793 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(12957) version 11:1:1:4 [ CSSD]2013-05-23 17:08:27.800 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:08:27.800 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:27.800 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:08:27.800 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:27.800 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 158 (0x7f431bfc3f10) [ CSSD]2013-05-23 17:08:27.800 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3f10 (RPC#158) tag(9e002d) sent to node 1 [ CSSD]2013-05-23 17:08:27.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(009e002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:27.800 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015035, gin 1, member 0 [ CSSD]2013-05-23 17:08:27.800 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:08:27.801 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:08:27.801 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:08:27.801 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3f10 (RPC#158) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:27.801 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc3f10, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:27.801 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:08:27.801 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 158 [ CSSD]2013-05-23 17:08:27.807 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:27.807 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:27.807 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:27.807 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 159 (0x7f431bfc3f90) [ CSSD]2013-05-23 17:08:27.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc3f90 (RPC#159) tag(9f002d) sent to node 1 [ CSSD]2013-05-23 17:08:27.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(009f002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:27.808 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:27.808 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc3f90 (RPC#159) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:27.808 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc3f90, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:27.808 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 159 [ CSSD]2013-05-23 17:08:27.808 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015036, gin 1, member 0 [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:08:27.848 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:08:27.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:28.010 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:08:28.010 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 12957 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:08:28.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:28.052 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:28.717 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:08:28.717 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 160 (0x7f431bfc4010) [ CSSD]2013-05-23 17:08:28.717 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4010 (RPC#160) tag(a0002d) sent to node 1 [ CSSD]2013-05-23 17:08:28.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:28.718 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 160 (0x7f431bfc4010) [ CSSD]2013-05-23 17:08:28.718 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4010 (RPC#160) state 6, flags 0x202 [ CSSD]2013-05-23 17:08:28.718 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4010, ret 0, client (nil) [ CSSD]2013-05-23 17:08:28.718 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 160 [ CSSD]2013-05-23 17:08:29.330 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:29.330 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:08:29.912 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:30.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:30.063 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:31.676 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:08:31.677 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:08:31.678 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:08:31.678 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(13027) version 11:1:1:4 [ CSSD]2013-05-23 17:08:31.685 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:08:31.685 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:31.685 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:08:31.685 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:31.685 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 161 (0x7f431bfc4090) [ CSSD]2013-05-23 17:08:31.685 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4090 (RPC#161) tag(a1002d) sent to node 1 [ CSSD]2013-05-23 17:08:31.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00a1002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:31.685 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015037, gin 1, member 0 [ CSSD]2013-05-23 17:08:31.685 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:08:31.686 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:08:31.686 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:08:31.686 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4090 (RPC#161) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:31.686 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4090, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:31.686 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:08:31.686 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 161 [ CSSD]2013-05-23 17:08:31.692 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:31.692 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:31.692 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:31.692 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 162 (0x7f431bfc4110) [ CSSD]2013-05-23 17:08:31.692 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4110 (RPC#162) tag(a2002d) sent to node 1 [ CSSD]2013-05-23 17:08:31.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00a2002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:31.693 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:31.693 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4110 (RPC#162) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:31.693 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4110, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:31.693 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 162 [ CSSD]2013-05-23 17:08:31.693 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:08:31.895 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:08:31.895 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 13027 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:08:31.918 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:32.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:32.069 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:33.929 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:34.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:34.095 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:34.333 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:34.334 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:35.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:36.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:36.167 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:37.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:37.889 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015038, gin 1, member 0 [ CSSD]2013-05-23 17:08:37.889 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:08:37.889 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:08:37.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:37.895 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:08:37.895 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:08:37.987 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:38.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:38.173 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:38.737 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:08:38.737 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 163 (0x7f431bfc4190) [ CSSD]2013-05-23 17:08:38.737 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4190 (RPC#163) tag(a3002d) sent to node 1 [ CSSD]2013-05-23 17:08:38.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:38.738 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 163 (0x7f431bfc4190) [ CSSD]2013-05-23 17:08:38.738 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4190 (RPC#163) state 6, flags 0x202 [ CSSD]2013-05-23 17:08:38.738 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4190, ret 0, client (nil) [ CSSD]2013-05-23 17:08:38.738 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 163 [ CSSD]2013-05-23 17:08:39.337 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:39.337 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:40.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:40.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:40.184 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:41.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:41.360 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015039, gin 1, member 0 [ CSSD]2013-05-23 17:08:41.360 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:08:41.360 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:08:41.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:41.367 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:08:41.367 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:08:41.492 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:08:41.493 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:08:41.494 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:08:41.494 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:08:41.496 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:08:41.496 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:08:42.038 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:42.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:42.190 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:43.052 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:08:43.053 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:08:43.053 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:08:43.053 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(13128) version 11:1:1:4 [ CSSD]2013-05-23 17:08:43.060 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:08:43.060 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:43.060 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f3ef0 with con 0x22f2a10, requested num -1 [ CSSD]2013-05-23 17:08:43.060 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:43.060 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 164 (0x7f431bfc4210) [ CSSD]2013-05-23 17:08:43.060 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4210 (RPC#164) tag(a4002d) sent to node 1 [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00a4002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015040, gin 1, member 0 [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4210 (RPC#164) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4210, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:43.060 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f3ef0 [ CSSD]2013-05-23 17:08:43.061 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 164 [ CSSD]2013-05-23 17:08:43.067 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:08:43.067 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f3ef0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:43.067 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3cc0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:43.067 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 165 (0x7f431bfc4290) [ CSSD]2013-05-23 17:08:43.067 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4290 (RPC#165) tag(a5002d) sent to node 1 [ CSSD]2013-05-23 17:08:43.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00a5002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:43.068 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:43.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4290 (RPC#165) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:43.068 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4290, ret 0, client 0x22f3ef0 member 0x22f3cc0 [ CSSD]2013-05-23 17:08:43.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 165 [ CSSD]2013-05-23 17:08:43.068 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3cc0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:08:43.270 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:08:43.270 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid ospid 13128 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:08:44.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:44.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:44.236 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:44.341 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:44.341 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:46.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:46.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:46.268 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:48.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:48.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:48.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:48.343 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:48.344 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:08:48.767 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:08:48.767 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 166 (0x7f431bfc4310) [ CSSD]2013-05-23 17:08:48.767 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4310 (RPC#166) tag(a6002d) sent to node 1 [ CSSD]2013-05-23 17:08:48.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:48.768 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 166 (0x7f431bfc4310) [ CSSD]2013-05-23 17:08:48.768 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4310 (RPC#166) state 6, flags 0x202 [ CSSD]2013-05-23 17:08:48.768 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4310, ret 0, client (nil) [ CSSD]2013-05-23 17:08:48.768 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 166 [ CSSD]2013-05-23 17:08:50.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:50.294 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:50.294 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:52.118 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:52.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:52.300 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:53.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:53.140 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015041, gin 1, member 0 [ CSSD]2013-05-23 17:08:53.140 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:08:53.140 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:08:53.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:53.147 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:08:53.147 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:08:53.347 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:53.347 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:54.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:54.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:54.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:56.135 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:56.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:56.329 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:58.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:08:58.312 [1191196992] >TRACE: clssgmAllocProc: (0x22f2a10) allocated [ CSSD]2013-05-23 17:08:58.313 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:08:58.314 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2a10 - 1,2,3 [ CSSD]2013-05-23 17:08:58.314 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f37a0) proc(0x22f2a10) pid(13284) version 11:1:1:4 [ CSSD]2013-05-23 17:08:58.320 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f2a10), client(1/0x22f1d30) [ CSSD]2013-05-23 17:08:58.320 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:08:58.320 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22f3e00, requested num -1 [ CSSD]2013-05-23 17:08:58.320 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:58.320 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 167 (0x7f431bfc4390) [ CSSD]2013-05-23 17:08:58.320 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4390 (RPC#167) tag(a7002d) sent to node 1 [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00a7002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015042, gin 1, member 0 [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f4930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4390 (RPC#167) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4390, ret 0, client 0x22f1d30 member 0x22f2ff0 [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:08:58.321 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 167 [ CSSD]2013-05-23 17:08:58.328 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:08:58.328 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:58.328 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2ff0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:08:58.328 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 168 (0x7f431bfc4410) [ CSSD]2013-05-23 17:08:58.328 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4410 (RPC#168) tag(a8002d) sent to node 1 [ CSSD]2013-05-23 17:08:58.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00a8002d) incarnation(174631051) [ CSSD]2013-05-23 17:08:58.328 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:08:58.328 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4410 (RPC#168) state 6, flags 0x100 [ CSSD]2013-05-23 17:08:58.328 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4410, ret 0, client 0x22f1d30 member 0x22f2ff0 [ CSSD]2013-05-23 17:08:58.328 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 168 [ CSSD]2013-05-23 17:08:58.328 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2ff0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:08:58.350 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:08:58.350 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:08:58.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:58.368 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:08:58.531 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2a10 [ CSSD]2013-05-23 17:08:58.531 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2a10) con(0x22f37a0) skgpid ospid 13284 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:08:58.786 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:08:58.787 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 169 (0x7f431bfc4490) [ CSSD]2013-05-23 17:08:58.787 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4490 (RPC#169) tag(a9002d) sent to node 1 [ CSSD]2013-05-23 17:08:58.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:08:58.787 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 169 (0x7f431bfc4490) [ CSSD]2013-05-23 17:08:58.787 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4490 (RPC#169) state 6, flags 0x202 [ CSSD]2013-05-23 17:08:58.787 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4490, ret 0, client (nil) [ CSSD]2013-05-23 17:08:58.788 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 169 [ CSSD]2013-05-23 17:09:00.127 [1191196992] >TRACE: clssgmAllocProc: (0x22f3dc0) allocated [ CSSD]2013-05-23 17:09:00.127 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:09:00.128 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3dc0 - 1,2,3 [ CSSD]2013-05-23 17:09:00.128 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d71f0) proc(0x22f3dc0) pid(13258/13258) version 11:1:1:4 [ CSSD]2013-05-23 17:09:00.134 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22f3dc0), client(1/0x22f3ef0) [ CSSD]2013-05-23 17:09:00.134 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:09:00.134 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:09:00.134 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:09:00.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:00.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:00.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:02.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:02.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:02.391 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:03.353 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:03.354 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:04.229 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:04.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:04.403 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:06.235 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:06.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:06.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:07.356 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:07.357 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:09:08.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:08.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:08.390 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015043, gin 1, member 0 [ CSSD]2013-05-23 17:09:08.390 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:09:08.390 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:09:08.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:08.397 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:09:08.397 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:09:08.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:08.415 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:08.807 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:09:08.807 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 170 (0x7f431bfc4510) [ CSSD]2013-05-23 17:09:08.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4510 (RPC#170) tag(aa002d) sent to node 1 [ CSSD]2013-05-23 17:09:08.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:08.808 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 170 (0x7f431bfc4510) [ CSSD]2013-05-23 17:09:08.808 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4510 (RPC#170) state 6, flags 0x202 [ CSSD]2013-05-23 17:09:08.808 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4510, ret 0, client (nil) [ CSSD]2013-05-23 17:09:08.808 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 170 [ CSSD]2013-05-23 17:09:10.294 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:10.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:10.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:12.299 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:12.360 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:12.360 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:12.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:12.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:13.573 [1191196992] >TRACE: clssgmAllocProc: (0x22f4f00) allocated [ CSSD]2013-05-23 17:09:13.574 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:09:13.575 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4f00 - 1,2,3 [ CSSD]2013-05-23 17:09:13.575 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2e50) proc(0x22f4f00) pid(13439) version 11:1:1:4 [ CSSD]2013-05-23 17:09:13.591 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f4f00), client(1/0x22f1d30) [ CSSD]2013-05-23 17:09:13.591 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:09:13.591 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22ed1f0, requested num -1 [ CSSD]2013-05-23 17:09:13.591 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:13.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 171 (0x7f431bfc4590) [ CSSD]2013-05-23 17:09:13.591 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4590 (RPC#171) tag(ab002d) sent to node 1 [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00ab002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015044, gin 1, member 0 [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed7b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4590 (RPC#171) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4590, ret 0, client 0x22f1d30 member 0x22f4e00 [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:09:13.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 171 [ CSSD]2013-05-23 17:09:13.599 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:09:13.599 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:13.599 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4e00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:13.599 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 172 (0x7f431bfc4610) [ CSSD]2013-05-23 17:09:13.599 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4610 (RPC#172) tag(ac002d) sent to node 1 [ CSSD]2013-05-23 17:09:13.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00ac002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:13.600 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:13.600 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4610 (RPC#172) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:13.600 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4610, ret 0, client 0x22f1d30 member 0x22f4e00 [ CSSD]2013-05-23 17:09:13.600 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 172 [ CSSD]2013-05-23 17:09:13.600 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4e00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:09:13.802 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4f00 [ CSSD]2013-05-23 17:09:13.802 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4f00) con(0x22f2e50) skgpid ospid 13439 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:09:14.311 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:14.484 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:14.484 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:16.317 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:16.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:16.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:17.120 [1191196992] >TRACE: clssgmAllocProc: (0x22f4f00) allocated [ CSSD]2013-05-23 17:09:17.121 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:09:17.121 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f4f00 - 1,2,3 [ CSSD]2013-05-23 17:09:17.121 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2e50) proc(0x22f4f00) pid(13256/13256) version 11:1:1:4 [ CSSD]2013-05-23 17:09:17.138 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f4f00), client(1/0x22f1d30) [ CSSD]2013-05-23 17:09:17.138 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:09:17.138 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:09:17.138 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:09:17.363 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:17.363 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:18.409 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:18.494 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:18.494 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:18.848 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:09:18.848 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 173 (0x7f431bfc4690) [ CSSD]2013-05-23 17:09:18.848 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4690 (RPC#173) tag(ad002d) sent to node 1 [ CSSD]2013-05-23 17:09:18.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:18.849 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 173 (0x7f431bfc4690) [ CSSD]2013-05-23 17:09:18.849 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4690 (RPC#173) state 6, flags 0x202 [ CSSD]2013-05-23 17:09:18.849 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4690, ret 0, client (nil) [ CSSD]2013-05-23 17:09:18.849 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 173 [ CSSD]2013-05-23 17:09:20.415 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:20.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:20.506 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:22.366 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:22.367 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:22.420 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:22.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:22.511 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:23.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:23.640 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015045, gin 1, member 0 [ CSSD]2013-05-23 17:09:23.640 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:09:23.640 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:09:23.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:23.647 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:09:23.647 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:09:24.441 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:24.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:24.563 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:26.369 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:26.370 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:09:26.447 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:26.569 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:26.569 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:28.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.102 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015046, gin 1, member 0 [ CSSD]2013-05-23 17:09:28.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:09:28.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:09:28.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.109 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:09:28.109 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:09:28.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:09:28.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:09:28.404 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:09:28.404 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:09:28.499 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:28.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.575 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:28.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.575 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015047, gin 1, member 0 [ CSSD]2013-05-23 17:09:28.575 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c0958d0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:09:28.575 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:09:28.583 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.583 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:09:28.583 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:09:28.838 [1191196992] >TRACE: clssgmAllocProc: (0x22e4fc0) allocated [ CSSD]2013-05-23 17:09:28.839 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:09:28.839 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4fc0 - 1,2,3 [ CSSD]2013-05-23 17:09:28.839 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22eda10) proc(0x22e4fc0) pid(13581) version 11:1:1:4 [ CSSD]2013-05-23 17:09:28.846 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4fc0), client(1/0x22edfc0) [ CSSD]2013-05-23 17:09:28.846 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22edfc0) [ CSSD]2013-05-23 17:09:28.846 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22edfc0 with con 0x22e51e0, requested num -1 [ CSSD]2013-05-23 17:09:28.846 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:28.846 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 174 (0x7f431bfc4710) [ CSSD]2013-05-23 17:09:28.846 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4710 (RPC#174) tag(ae002d) sent to node 1 [ CSSD]2013-05-23 17:09:28.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00ae002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.846 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015048, gin 1, member 0 [ CSSD]2013-05-23 17:09:28.846 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23214a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:09:28.847 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:09:28.847 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:09:28.847 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4710 (RPC#174) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:28.847 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4710, ret 0, client 0x22edfc0 member 0x22e4eb0 [ CSSD]2013-05-23 17:09:28.847 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22edfc0 [ CSSD]2013-05-23 17:09:28.847 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 174 [ CSSD]2013-05-23 17:09:28.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22edfc0) [ CSSD]2013-05-23 17:09:28.854 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22edfc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:28.854 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:28.854 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 175 (0x7f431bfc4790) [ CSSD]2013-05-23 17:09:28.854 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4790 (RPC#175) tag(af002d) sent to node 1 [ CSSD]2013-05-23 17:09:28.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00af002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.854 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:28.854 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4790 (RPC#175) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:28.854 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4790, ret 0, client 0x22edfc0 member 0x22e4eb0 [ CSSD]2013-05-23 17:09:28.855 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 175 [ CSSD]2013-05-23 17:09:28.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:09:28.878 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:09:28.878 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 176 (0x7f431bfc4810) [ CSSD]2013-05-23 17:09:28.878 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4810 (RPC#176) tag(b0002d) sent to node 1 [ CSSD]2013-05-23 17:09:28.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:28.879 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 176 (0x7f431bfc4810) [ CSSD]2013-05-23 17:09:28.879 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4810 (RPC#176) state 6, flags 0x202 [ CSSD]2013-05-23 17:09:28.879 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4810, ret 0, client (nil) [ CSSD]2013-05-23 17:09:28.879 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 176 [ CSSD]2013-05-23 17:09:29.057 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4fc0 [ CSSD]2013-05-23 17:09:29.057 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4fc0) con(0x22eda10) skgpid ospid 13581 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:09:30.505 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:30.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:30.586 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:31.373 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:31.373 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:31.930 [1191196992] >TRACE: clssgmAllocProc: (0x22e4ef0) allocated [ CSSD]2013-05-23 17:09:31.931 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:09:31.932 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4ef0 - 1,2,3 [ CSSD]2013-05-23 17:09:31.932 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22edaf0) proc(0x22e4ef0) pid(13655) version 11:1:1:4 [ CSSD]2013-05-23 17:09:31.939 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4ef0), client(1/0x23213d0) [ CSSD]2013-05-23 17:09:31.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23213d0) [ CSSD]2013-05-23 17:09:31.939 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23213d0 with con 0x22ee170, requested num -1 [ CSSD]2013-05-23 17:09:31.939 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:31.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 177 (0x7f431bfc4890) [ CSSD]2013-05-23 17:09:31.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4890 (RPC#177) tag(b1002d) sent to node 1 [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00b1002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015049, gin 1, member 0 [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4890 (RPC#177) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4890, ret 0, client 0x23213d0 member 0x22e4de0 [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23213d0 [ CSSD]2013-05-23 17:09:31.940 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 177 [ CSSD]2013-05-23 17:09:31.946 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23213d0) [ CSSD]2013-05-23 17:09:31.946 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23213d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:31.946 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4de0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:31.946 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 178 (0x7f431bfc4910) [ CSSD]2013-05-23 17:09:31.946 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4910 (RPC#178) tag(b2002d) sent to node 1 [ CSSD]2013-05-23 17:09:31.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00b2002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:31.947 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:31.947 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4910 (RPC#178) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:31.947 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4910, ret 0, client 0x23213d0 member 0x22e4de0 [ CSSD]2013-05-23 17:09:31.947 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 178 [ CSSD]2013-05-23 17:09:31.947 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4de0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:09:32.149 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4ef0 [ CSSD]2013-05-23 17:09:32.149 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4ef0) con(0x22edaf0) skgpid ospid 13655 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:09:32.511 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:32.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:32.592 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:34.523 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:34.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:34.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:36.376 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:36.377 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:36.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:36.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:36.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:38.540 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:38.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:38.636 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:38.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:38.894 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015050, gin 1, member 0 [ CSSD]2013-05-23 17:09:38.894 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:09:38.894 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:09:38.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:38.901 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:09:38.901 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:09:38.908 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:09:38.908 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 179 (0x7f431bfc4990) [ CSSD]2013-05-23 17:09:38.909 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4990 (RPC#179) tag(b3002d) sent to node 1 [ CSSD]2013-05-23 17:09:38.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:38.909 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 179 (0x7f431bfc4990) [ CSSD]2013-05-23 17:09:38.909 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4990 (RPC#179) state 6, flags 0x202 [ CSSD]2013-05-23 17:09:38.909 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4990, ret 0, client (nil) [ CSSD]2013-05-23 17:09:38.909 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 179 [ CSSD]2013-05-23 17:09:40.547 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:40.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:40.686 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:41.380 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:41.380 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:42.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:42.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:42.691 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:44.105 [1191196992] >TRACE: clssgmAllocProc: (0x22e4fc0) allocated [ CSSD]2013-05-23 17:09:44.106 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:09:44.107 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4fc0 - 1,2,3 [ CSSD]2013-05-23 17:09:44.107 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed7e0) proc(0x22e4fc0) pid(13761) version 11:1:1:4 [ CSSD]2013-05-23 17:09:44.113 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4fc0), client(1/0x22e5240) [ CSSD]2013-05-23 17:09:44.113 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5240) [ CSSD]2013-05-23 17:09:44.113 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5240 with con 0x2320f80, requested num -1 [ CSSD]2013-05-23 17:09:44.113 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:44.113 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 180 (0x7f431bfc4a10) [ CSSD]2013-05-23 17:09:44.113 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4a10 (RPC#180) tag(b4002d) sent to node 1 [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00b4002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015051, gin 1, member 0 [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321550/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4a10 (RPC#180) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4a10, ret 0, client 0x22e5240 member 0x22e4eb0 [ CSSD]2013-05-23 17:09:44.113 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5240 [ CSSD]2013-05-23 17:09:44.114 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 180 [ CSSD]2013-05-23 17:09:44.120 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5240) [ CSSD]2013-05-23 17:09:44.120 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5240), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:44.120 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:44.120 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 181 (0x7f431bfc4a90) [ CSSD]2013-05-23 17:09:44.120 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4a90 (RPC#181) tag(b5002d) sent to node 1 [ CSSD]2013-05-23 17:09:44.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00b5002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:44.121 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:44.121 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4a90 (RPC#181) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:44.121 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4a90, ret 0, client 0x22e5240 member 0x22e4eb0 [ CSSD]2013-05-23 17:09:44.121 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 181 [ CSSD]2013-05-23 17:09:44.121 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:09:44.323 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4fc0 [ CSSD]2013-05-23 17:09:44.323 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4fc0) con(0x22ed7e0) skgpid ospid 13761 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:09:44.603 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:44.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:44.704 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:45.383 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:45.383 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:09:46.610 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:46.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:46.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:48.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:48.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:48.715 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:48.938 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:09:48.938 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 182 (0x7f431bfc4b10) [ CSSD]2013-05-23 17:09:48.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4b10 (RPC#182) tag(b6002d) sent to node 1 [ CSSD]2013-05-23 17:09:48.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:48.939 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 182 (0x7f431bfc4b10) [ CSSD]2013-05-23 17:09:48.939 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4b10 (RPC#182) state 6, flags 0x202 [ CSSD]2013-05-23 17:09:48.939 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4b10, ret 0, client (nil) [ CSSD]2013-05-23 17:09:48.939 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 182 [ CSSD]2013-05-23 17:09:50.386 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:50.387 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:50.628 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:50.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:50.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:52.633 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:52.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:52.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:54.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:54.143 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015052, gin 1, member 0 [ CSSD]2013-05-23 17:09:54.143 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:09:54.143 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:09:54.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:54.150 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:09:54.150 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:09:54.644 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:54.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:54.784 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:55.390 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:09:55.391 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:09:56.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:56.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:56.789 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:58.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:09:58.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:58.816 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:09:58.958 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:09:58.958 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 183 (0x7f431bfc4b90) [ CSSD]2013-05-23 17:09:58.958 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4b90 (RPC#183) tag(b7002d) sent to node 1 [ CSSD]2013-05-23 17:09:58.959 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:09:58.959 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 183 (0x7f431bfc4b90) [ CSSD]2013-05-23 17:09:58.959 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4b90 (RPC#183) state 6, flags 0x202 [ CSSD]2013-05-23 17:09:58.959 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4b90, ret 0, client (nil) [ CSSD]2013-05-23 17:09:58.959 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 183 [ CSSD]2013-05-23 17:09:59.356 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e6d60) allocated [ CSSD]2013-05-23 17:09:59.357 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:09:59.357 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e6d60 - 1,2,3 [ CSSD]2013-05-23 17:09:59.357 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0e6d60) pid(13909) version 11:1:1:4 [ CSSD]2013-05-23 17:09:59.363 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0e6d60), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:09:59.364 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:09:59.364 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 17:09:59.364 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:59.364 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 184 (0x7f431bfc4c10) [ CSSD]2013-05-23 17:09:59.364 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4c10 (RPC#184) tag(b8002d) sent to node 1 [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00b8002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015053, gin 1, member 0 [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4c10 (RPC#184) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4c10, ret 0, client 0x7f431c0cf720 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:09:59.364 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:09:59.365 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 184 [ CSSD]2013-05-23 17:09:59.371 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:09:59.371 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:59.371 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:09:59.371 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 185 (0x7f431bfc4c90) [ CSSD]2013-05-23 17:09:59.371 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4c90 (RPC#185) tag(b9002d) sent to node 1 [ CSSD]2013-05-23 17:09:59.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00b9002d) incarnation(174631051) [ CSSD]2013-05-23 17:09:59.372 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:09:59.372 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4c90 (RPC#185) state 6, flags 0x100 [ CSSD]2013-05-23 17:09:59.372 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4c90, ret 0, client 0x7f431c0cf720 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:09:59.372 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 185 [ CSSD]2013-05-23 17:09:59.372 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:09:59.574 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e6d60 [ CSSD]2013-05-23 17:09:59.574 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e6d60) con(0x7f431c12f0c0) skgpid ospid 13909 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:00.394 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:00.394 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:00.707 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:00.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:00.887 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:02.713 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:02.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:02.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:03.630 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e5840) allocated [ CSSD]2013-05-23 17:10:03.631 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:10:03.632 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e5840 - 1,2,3 [ CSSD]2013-05-23 17:10:03.632 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c0e5840) pid(12568/12568) version 11:1:1:4 [ CSSD]2013-05-23 17:10:03.637 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0e5840), client(1/0x7f431c108e70) [ CSSD]2013-05-23 17:10:03.637 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:10:03.637 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:10:03.637 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:10:04.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:04.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:04.904 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:05.397 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:05.398 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:06.750 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:06.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:06.910 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:08.012 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 17:10:08.013 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:10:08.013 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 17:10:08.013 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c12f520) pid(14201) version 11:1:1:4 [ CSSD]2013-05-23 17:10:08.604 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 17:10:08.604 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c0e5200) skgpid ospid 14201 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:08.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:08.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:08.915 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:08.979 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:10:08.979 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 186 (0x7f431bfc4d10) [ CSSD]2013-05-23 17:10:08.980 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4d10 (RPC#186) tag(ba002d) sent to node 1 [ CSSD]2013-05-23 17:10:08.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:08.980 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 186 (0x7f431bfc4d10) [ CSSD]2013-05-23 17:10:08.980 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4d10 (RPC#186) state 6, flags 0x202 [ CSSD]2013-05-23 17:10:08.980 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4d10, ret 0, client (nil) [ CSSD]2013-05-23 17:10:08.980 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 186 [ CSSD]2013-05-23 17:10:09.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:09.394 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015054, gin 1, member 0 [ CSSD]2013-05-23 17:10:09.394 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:10:09.394 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:10:09.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:09.400 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:10:09.400 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e6d60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:10:10.401 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:10.401 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:10.808 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:10.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:10.918 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:12.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:12.924 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:12.924 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:14.609 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12f520) allocated [ CSSD]2013-05-23 17:10:14.610 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:10:14.610 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12f520 - 1,2,3 [ CSSD]2013-05-23 17:10:14.610 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c12f520) pid(14352) version 11:1:1:4 [ CSSD]2013-05-23 17:10:14.617 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c12f520), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:10:14.617 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:10:14.617 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 17:10:14.617 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:14.617 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 187 (0x7f431bfc4d90) [ CSSD]2013-05-23 17:10:14.617 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4d90 (RPC#187) tag(bb002d) sent to node 1 [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00bb002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015055, gin 1, member 0 [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4d90 (RPC#187) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4d90, ret 0, client 0x7f431c109f80 member 0x7f431c095670 [ CSSD]2013-05-23 17:10:14.617 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:10:14.618 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 187 [ CSSD]2013-05-23 17:10:14.624 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:10:14.624 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:14.624 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095670) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:14.624 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 188 (0x7f431bfc4e10) [ CSSD]2013-05-23 17:10:14.624 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4e10 (RPC#188) tag(bc002d) sent to node 1 [ CSSD]2013-05-23 17:10:14.624 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00bc002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:14.624 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:14.624 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4e10 (RPC#188) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:14.624 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc4e10, ret 0, client 0x7f431c109f80 member 0x7f431c095670 [ CSSD]2013-05-23 17:10:14.625 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 188 [ CSSD]2013-05-23 17:10:14.625 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095670) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:10:14.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:14.827 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12f520 [ CSSD]2013-05-23 17:10:14.827 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12f520) con(0x7f431c0e5200) skgpid ospid 14352 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:14.976 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:14.976 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:15.405 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:15.405 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:16.829 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:16.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:16.981 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:18.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:18.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:18.986 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:18.999 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:10:19.000 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 189 (0x7f431bfc4e90) [ CSSD]2013-05-23 17:10:19.000 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4e90 (RPC#189) tag(bd002d) sent to node 1 [ CSSD]2013-05-23 17:10:19.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:19.001 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 189 (0x7f431bfc4e90) [ CSSD]2013-05-23 17:10:19.001 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4e90 (RPC#189) state 6, flags 0x202 [ CSSD]2013-05-23 17:10:19.001 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4e90, ret 0, client (nil) [ CSSD]2013-05-23 17:10:19.001 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 189 [ CSSD]2013-05-23 17:10:19.389 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0e5840), client(2/0x7f431c109f80) [ CSSD]2013-05-23 17:10:19.389 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c109f80) [ CSSD]2013-05-23 17:10:19.389 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:10:19.389 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 17:10:19.408 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:19.408 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:10:19.970 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22dabd0) [ CSSD]2013-05-23 17:10:19.970 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22dabd0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:10:19.970 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22dabd0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:10:19.970 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:10:19.970 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc5a0 [ CSSD]2013-05-23 17:10:19.970 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc5a0) con(0x22e02b0) skgpid 8942 ospid 8942 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:20.849 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:20.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:20.999 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:22.036 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 17:10:22.037 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:10:22.038 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 17:10:22.038 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c13ea10) pid(14439/14439) version 11:1:1:4 [ CSSD]2013-05-23 17:10:22.054 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c13ea10), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:10:22.054 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:10:22.054 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:10:22.054 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:10:22.360 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f4f00), client(2/0x7f431c135120) [ CSSD]2013-05-23 17:10:22.360 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c135120) [ CSSD]2013-05-23 17:10:22.360 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:10:22.360 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 17:10:22.753 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c13ea10), client(2/0x7f431c0cf720) [ CSSD]2013-05-23 17:10:22.753 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 17:10:22.753 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:10:22.753 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:10:22.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:23.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:23.005 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:24.411 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:24.411 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:24.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:24.654 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015056, gin 1, member 0 [ CSSD]2013-05-23 17:10:24.654 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:10:24.654 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:10:24.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:24.661 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:10:24.661 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:10:24.907 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:25.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:25.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:26.913 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:27.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:27.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:28.370 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:28.370 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015057, gin 1, member 0 [ CSSD]2013-05-23 17:10:28.370 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:10:28.370 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:10:28.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:28.377 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:10:28.377 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:10:29.003 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:29.019 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:10:29.019 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 190 (0x7f431bfc4f10) [ CSSD]2013-05-23 17:10:29.020 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4f10 (RPC#190) tag(be002d) sent to node 1 [ CSSD]2013-05-23 17:10:29.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:29.020 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 190 (0x7f431bfc4f10) [ CSSD]2013-05-23 17:10:29.020 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4f10 (RPC#190) state 6, flags 0x202 [ CSSD]2013-05-23 17:10:29.020 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc4f10, ret 0, client (nil) [ CSSD]2013-05-23 17:10:29.020 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 190 [ CSSD]2013-05-23 17:10:29.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:29.066 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:29.415 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:29.416 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:29.859 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1088b0) allocated [ CSSD]2013-05-23 17:10:29.860 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:10:29.861 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1088b0 - 1,2,3 [ CSSD]2013-05-23 17:10:29.861 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c10a550) proc(0x7f431c1088b0) pid(14511) version 11:1:1:4 [ CSSD]2013-05-23 17:10:29.879 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c1088b0), client(1/0x7f431c095b30) [ CSSD]2013-05-23 17:10:29.879 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 17:10:29.879 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095b30 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:10:29.879 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:29.879 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 191 (0x7f431bfc4f90) [ CSSD]2013-05-23 17:10:29.879 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc4f90 (RPC#191) tag(bf002d) sent to node 1 [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00bf002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015058, gin 1, member 0 [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c129640/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc4f90 (RPC#191) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc4f90, ret 0, client 0x7f431c095b30 member 0x7f431c1086b0 [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095b30 [ CSSD]2013-05-23 17:10:29.880 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 191 [ CSSD]2013-05-23 17:10:29.888 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095b30) [ CSSD]2013-05-23 17:10:29.888 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095b30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:29.888 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1086b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:29.888 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 192 (0x7f431bfc5010) [ CSSD]2013-05-23 17:10:29.888 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5010 (RPC#192) tag(c0002d) sent to node 1 [ CSSD]2013-05-23 17:10:29.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00c0002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:29.889 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:29.889 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5010 (RPC#192) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:29.889 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5010, ret 0, client 0x7f431c095b30 member 0x7f431c1086b0 [ CSSD]2013-05-23 17:10:29.889 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 192 [ CSSD]2013-05-23 17:10:29.889 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1086b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:10:30.091 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1088b0 [ CSSD]2013-05-23 17:10:30.091 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1088b0) con(0x7f431c10a550) skgpid ospid 14511 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:31.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:31.080 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:31.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:32.197 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:10:32.198 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:10:32.199 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:10:32.199 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1215d0) proc(0x7f431c13ef60) pid(14564) version 11:1:1:4 [ CSSD]2013-05-23 17:10:32.205 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ef60), client(1/0x22e02b0) [ CSSD]2013-05-23 17:10:32.205 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e02b0) [ CSSD]2013-05-23 17:10:32.205 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e02b0 with con 0x231d790, requested num -1 [ CSSD]2013-05-23 17:10:32.205 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:32.205 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 193 (0x7f431bfc5090) [ CSSD]2013-05-23 17:10:32.205 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5090 (RPC#193) tag(c1002d) sent to node 1 [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00c1002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015059, gin 1, member 0 [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5090 (RPC#193) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5090, ret 0, client 0x22e02b0 member 0x22d7750 [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e02b0 [ CSSD]2013-05-23 17:10:32.206 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 193 [ CSSD]2013-05-23 17:10:32.213 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e02b0) [ CSSD]2013-05-23 17:10:32.213 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e02b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:32.213 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:32.213 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 194 (0x7f431bfc5110) [ CSSD]2013-05-23 17:10:32.213 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5110 (RPC#194) tag(c2002d) sent to node 1 [ CSSD]2013-05-23 17:10:32.213 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00c2002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:32.213 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:32.213 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5110 (RPC#194) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:32.213 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5110, ret 0, client 0x22e02b0 member 0x22d7750 [ CSSD]2013-05-23 17:10:32.213 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 194 [ CSSD]2013-05-23 17:10:32.213 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:10:32.416 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:10:32.416 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c1215d0) skgpid ospid 14564 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:33.086 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:33.086 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:33.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:34.419 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:34.420 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:35.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:35.097 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:35.107 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:37.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:37.104 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:37.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:39.049 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:10:39.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 195 (0x7f431bfc5190) [ CSSD]2013-05-23 17:10:39.050 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5190 (RPC#195) tag(c3002d) sent to node 1 [ CSSD]2013-05-23 17:10:39.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:39.050 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 195 (0x7f431bfc5190) [ CSSD]2013-05-23 17:10:39.050 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5190 (RPC#195) state 6, flags 0x202 [ CSSD]2013-05-23 17:10:39.050 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5190, ret 0, client (nil) [ CSSD]2013-05-23 17:10:39.050 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 195 [ CSSD]2013-05-23 17:10:39.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:39.128 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:39.204 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:39.423 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:39.423 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:39.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:39.912 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015060, gin 1, member 0 [ CSSD]2013-05-23 17:10:39.912 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:10:39.912 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:10:39.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:39.920 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:10:39.920 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:10:41.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:41.220 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:41.290 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:43.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:43.226 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:43.321 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:44.427 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:44.427 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:45.129 [1191196992] >TRACE: clssgmAllocProc: (0x22ec6a0) allocated [ CSSD]2013-05-23 17:10:45.130 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:10:45.131 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ec6a0 - 1,2,3 [ CSSD]2013-05-23 17:10:45.131 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7820) proc(0x22ec6a0) pid(14661) version 11:1:1:4 [ CSSD]2013-05-23 17:10:45.137 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ec6a0), client(1/0x22dabc0) [ CSSD]2013-05-23 17:10:45.137 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:10:45.137 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabc0 with con 0x22ff740, requested num -1 [ CSSD]2013-05-23 17:10:45.137 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:45.137 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 196 (0x7f431bfc5210) [ CSSD]2013-05-23 17:10:45.137 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5210 (RPC#196) tag(c4002d) sent to node 1 [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00c4002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015061, gin 1, member 0 [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed900/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5210 (RPC#196) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:45.137 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5210, ret 0, client 0x22dabc0 member 0x22fc690 [ CSSD]2013-05-23 17:10:45.138 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 17:10:45.138 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 196 [ CSSD]2013-05-23 17:10:45.144 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:10:45.144 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:45.144 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc690) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:45.144 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 197 (0x7f431bfc5290) [ CSSD]2013-05-23 17:10:45.144 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5290 (RPC#197) tag(c5002d) sent to node 1 [ CSSD]2013-05-23 17:10:45.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00c5002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:45.145 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:45.145 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5290 (RPC#197) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:45.145 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5290, ret 0, client 0x22dabc0 member 0x22fc690 [ CSSD]2013-05-23 17:10:45.145 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 197 [ CSSD]2013-05-23 17:10:45.145 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc690) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:10:45.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:45.278 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:45.328 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:45.347 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ec6a0 [ CSSD]2013-05-23 17:10:45.347 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ec6a0) con(0x22d7820) skgpid ospid 14661 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:47.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:47.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:47.335 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:49.079 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:10:49.079 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 198 (0x7f431bfc5310) [ CSSD]2013-05-23 17:10:49.079 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5310 (RPC#198) tag(c6002d) sent to node 1 [ CSSD]2013-05-23 17:10:49.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:49.080 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 198 (0x7f431bfc5310) [ CSSD]2013-05-23 17:10:49.080 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5310 (RPC#198) state 6, flags 0x202 [ CSSD]2013-05-23 17:10:49.080 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5310, ret 0, client (nil) [ CSSD]2013-05-23 17:10:49.080 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 198 [ CSSD]2013-05-23 17:10:49.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:49.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:49.355 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:49.431 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:49.431 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:51.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:51.322 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:51.361 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:53.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:53.327 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:53.412 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:53.434 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:53.434 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:10:55.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:55.164 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015062, gin 1, member 0 [ CSSD]2013-05-23 17:10:55.164 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:10:55.165 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:10:55.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:55.171 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:10:55.171 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:10:55.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:55.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:55.418 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:57.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:57.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:57.425 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:10:58.376 [1191196992] >TRACE: clssgmAllocProc: (0x22fc5a0) allocated [ CSSD]2013-05-23 17:10:58.377 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:10:58.377 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc5a0 - 1,2,3 [ CSSD]2013-05-23 17:10:58.377 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7820) proc(0x22fc5a0) pid(14799) version 11:1:1:4 [ CSSD]2013-05-23 17:10:58.384 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22fc5a0), client(1/0x22dabc0) [ CSSD]2013-05-23 17:10:58.384 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:10:58.384 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabc0 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 17:10:58.384 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:58.384 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 199 (0x7f431bfc5390) [ CSSD]2013-05-23 17:10:58.384 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5390 (RPC#199) tag(c7002d) sent to node 1 [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00c7002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015063, gin 1, member 0 [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed900/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5390 (RPC#199) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5390, ret 0, client 0x22dabc0 member 0x22ec790 [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 17:10:58.385 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 199 [ CSSD]2013-05-23 17:10:58.392 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:10:58.392 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:58.392 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ec790) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:10:58.392 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 200 (0x7f431bfc5410) [ CSSD]2013-05-23 17:10:58.392 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5410 (RPC#200) tag(c8002d) sent to node 1 [ CSSD]2013-05-23 17:10:58.392 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00c8002d) incarnation(174631051) [ CSSD]2013-05-23 17:10:58.392 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:10:58.392 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5410 (RPC#200) state 6, flags 0x100 [ CSSD]2013-05-23 17:10:58.392 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5410, ret 0, client 0x22dabc0 member 0x22ec790 [ CSSD]2013-05-23 17:10:58.393 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 200 [ CSSD]2013-05-23 17:10:58.393 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ec790) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:10:58.438 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:10:58.438 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:10:58.595 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc5a0 [ CSSD]2013-05-23 17:10:58.595 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc5a0) con(0x22d7820) skgpid ospid 14799 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:10:59.109 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:10:59.109 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 201 (0x7f431bfc5490) [ CSSD]2013-05-23 17:10:59.109 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5490 (RPC#201) tag(c9002d) sent to node 1 [ CSSD]2013-05-23 17:10:59.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:59.110 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 201 (0x7f431bfc5490) [ CSSD]2013-05-23 17:10:59.110 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5490 (RPC#201) state 6, flags 0x202 [ CSSD]2013-05-23 17:10:59.110 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5490, ret 0, client (nil) [ CSSD]2013-05-23 17:10:59.110 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 201 [ CSSD]2013-05-23 17:10:59.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:10:59.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:10:59.433 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:00.379 [1191196992] >TRACE: clssgmAllocProc: (0x22ec6a0) allocated [ CSSD]2013-05-23 17:11:00.381 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:11:00.382 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ec6a0 - 1,2,3 [ CSSD]2013-05-23 17:11:00.382 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7820) proc(0x22ec6a0) pid(14820) version 11:1:1:4 [ CSSD]2013-05-23 17:11:00.389 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ec6a0), client(1/0x22dabc0) [ CSSD]2013-05-23 17:11:00.389 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:11:00.389 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dabc0 with con 0x23037e0, requested num -1 [ CSSD]2013-05-23 17:11:00.389 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:00.389 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 202 (0x7f431bfc5510) [ CSSD]2013-05-23 17:11:00.389 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5510 (RPC#202) tag(ca002d) sent to node 1 [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00ca002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015064, gin 1, member 0 [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed900/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5510 (RPC#202) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5510, ret 0, client 0x22dabc0 member 0x22fc690 [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 17:11:00.390 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 202 [ CSSD]2013-05-23 17:11:00.397 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:11:00.397 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:00.397 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc690) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:00.397 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 203 (0x7f431bfc5590) [ CSSD]2013-05-23 17:11:00.397 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5590 (RPC#203) tag(cb002d) sent to node 1 [ CSSD]2013-05-23 17:11:00.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00cb002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:00.397 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:00.397 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5590 (RPC#203) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:00.397 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5590, ret 0, client 0x22dabc0 member 0x22fc690 [ CSSD]2013-05-23 17:11:00.397 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 203 [ CSSD]2013-05-23 17:11:00.397 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc690) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:11:00.600 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ec6a0 [ CSSD]2013-05-23 17:11:00.600 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ec6a0) con(0x22d7820) skgpid ospid 14820 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:01.420 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:01.420 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:01.439 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:03.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:03.426 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:03.440 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:03.441 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:03.452 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:05.459 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:05.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:05.480 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:07.465 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:07.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:07.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:08.444 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:08.444 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:09.129 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:11:09.129 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 204 (0x7f431bfc5610) [ CSSD]2013-05-23 17:11:09.129 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5610 (RPC#204) tag(cc002d) sent to node 1 [ CSSD]2013-05-23 17:11:09.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:09.130 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 204 (0x7f431bfc5610) [ CSSD]2013-05-23 17:11:09.130 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5610 (RPC#204) state 6, flags 0x202 [ CSSD]2013-05-23 17:11:09.130 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5610, ret 0, client (nil) [ CSSD]2013-05-23 17:11:09.130 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 204 [ CSSD]2013-05-23 17:11:09.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:09.496 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:09.516 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:10.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:10.415 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015065, gin 1, member 0 [ CSSD]2013-05-23 17:11:10.415 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:11:10.415 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:11:10.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:10.454 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:11:10.454 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:11:11.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:11.522 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:11.542 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:12.447 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:12.447 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:11:13.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:13.528 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:13.553 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:15.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:15.580 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:15.580 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:15.643 [1191196992] >TRACE: clssgmAllocProc: (0x22fc5a0) allocated [ CSSD]2013-05-23 17:11:15.643 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:11:15.644 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc5a0 - 1,2,3 [ CSSD]2013-05-23 17:11:15.644 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7820) proc(0x22fc5a0) pid(14978) version 11:1:1:4 [ CSSD]2013-05-23 17:11:15.649 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22fc5a0), client(1/0x22dab00) [ CSSD]2013-05-23 17:11:15.649 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dab00) [ CSSD]2013-05-23 17:11:15.649 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dab00 with con 0x22e02b0, requested num -1 [ CSSD]2013-05-23 17:11:15.649 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:15.650 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 205 (0x7f431bfc5690) [ CSSD]2013-05-23 17:11:15.650 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5690 (RPC#205) tag(cd002d) sent to node 1 [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00cd002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015066, gin 1, member 0 [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed830/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5690 (RPC#205) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5690, ret 0, client 0x22dab00 member 0x22ec790 [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dab00 [ CSSD]2013-05-23 17:11:15.650 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 205 [ CSSD]2013-05-23 17:11:15.658 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dab00) [ CSSD]2013-05-23 17:11:15.658 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dab00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:15.658 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ec790) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:15.658 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 206 (0x7f431bfc5710) [ CSSD]2013-05-23 17:11:15.658 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5710 (RPC#206) tag(ce002d) sent to node 1 [ CSSD]2013-05-23 17:11:15.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00ce002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:15.658 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:15.659 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5710 (RPC#206) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:15.659 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5710, ret 0, client 0x22dab00 member 0x22ec790 [ CSSD]2013-05-23 17:11:15.659 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 206 [ CSSD]2013-05-23 17:11:15.659 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ec790) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:11:15.862 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc5a0 [ CSSD]2013-05-23 17:11:15.862 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc5a0) con(0x22d7820) skgpid ospid 14978 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:17.450 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:17.451 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:17.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:17.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:17.645 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:19.150 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:11:19.150 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 207 (0x7f431bfc5790) [ CSSD]2013-05-23 17:11:19.151 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5790 (RPC#207) tag(cf002d) sent to node 1 [ CSSD]2013-05-23 17:11:19.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:19.151 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 207 (0x7f431bfc5790) [ CSSD]2013-05-23 17:11:19.151 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5790 (RPC#207) state 6, flags 0x202 [ CSSD]2013-05-23 17:11:19.151 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5790, ret 0, client (nil) [ CSSD]2013-05-23 17:11:19.151 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 207 [ CSSD]2013-05-23 17:11:19.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:19.617 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:19.637 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:20.306 [1191196992] >TRACE: clssgmAllocProc: (0x22ec6a0) allocated [ CSSD]2013-05-23 17:11:20.307 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:11:20.308 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ec6a0 - 1,2,3 [ CSSD]2013-05-23 17:11:20.308 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7820) proc(0x22ec6a0) pid(15083) version 11:1:1:4 [ CSSD]2013-05-23 17:11:20.315 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ec6a0), client(1/0x22dabc0) [ CSSD]2013-05-23 17:11:20.315 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:11:20.315 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22dabc0 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 17:11:20.315 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:11:20.315 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 208 (0x7f431bfc5810) [ CSSD]2013-05-23 17:11:20.315 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5810 (RPC#208) tag(d0002d) sent to node 1 [ CSSD]2013-05-23 17:11:20.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00d0002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:20.315 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015067, gin 1, member 0 [ CSSD]2013-05-23 17:11:20.315 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ed900/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:11:20.315 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:11:20.316 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:11:20.316 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5810 (RPC#208) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:20.316 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5810, ret 0, client 0x22dabc0 member 0x22fc690 [ CSSD]2013-05-23 17:11:20.316 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dabc0 [ CSSD]2013-05-23 17:11:20.316 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 208 [ CSSD]2013-05-23 17:11:20.323 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dabc0) [ CSSD]2013-05-23 17:11:20.323 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dabc0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:11:20.323 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc690) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:11:20.323 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 209 (0x7f431bfc5890) [ CSSD]2013-05-23 17:11:20.323 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5890 (RPC#209) tag(d1002d) sent to node 1 [ CSSD]2013-05-23 17:11:20.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00d1002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:20.323 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:11:20.323 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5890 (RPC#209) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:20.323 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5890, ret 0, client 0x22dabc0 member 0x22fc690 [ CSSD]2013-05-23 17:11:20.323 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 209 [ CSSD]2013-05-23 17:11:20.323 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22fc690) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:11:20.438 [1191196992] >TRACE: clssgmAllocProc: (0x22e4bb0) allocated [ CSSD]2013-05-23 17:11:20.440 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:11:20.440 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4bb0 - 1,2,3 [ CSSD]2013-05-23 17:11:20.440 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22fc690) proc(0x22e4bb0) pid(15088) version 11:1:1:4 [ CSSD]2013-05-23 17:11:20.454 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:11:20.456 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:11:20.459 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4bb0 [ CSSD]2013-05-23 17:11:20.459 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4bb0) con(0x22fc690) skgpid ospid 15088 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:20.526 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ec6a0 [ CSSD]2013-05-23 17:11:20.526 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ec6a0) con(0x22d7820) skgpid ospid 15083 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:21.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:21.624 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:21.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:22.454 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:22.454 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:23.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:23.629 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:23.653 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:24.586 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x205e360) [ CSSD]2013-05-23 17:11:24.586 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x205e360), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:11:24.586 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x205e360), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:11:24.586 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:11:24.596 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22fdc30) [ CSSD]2013-05-23 17:11:24.596 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22fdc30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:11:24.596 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22fdc30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:11:24.596 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:11:24.596 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302b40 [ CSSD]2013-05-23 17:11:24.596 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302b40) con(0x2302d20) skgpid 9741 ospid 9741 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:25.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:25.650 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:25.659 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:25.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:25.667 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015068, gin 1, member 0 [ CSSD]2013-05-23 17:11:25.667 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:11:25.667 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:11:25.706 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:25.706 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:11:25.706 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:11:27.457 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:27.458 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:27.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:27.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:27.666 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:28.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:28.626 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015069, gin 1, member 0 [ CSSD]2013-05-23 17:11:28.626 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:11:28.626 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:11:28.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:28.667 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:11:28.667 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:11:29.170 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:11:29.171 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 210 (0x7f431bfc5910) [ CSSD]2013-05-23 17:11:29.171 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5910 (RPC#210) tag(d2002d) sent to node 1 [ CSSD]2013-05-23 17:11:29.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:29.171 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 210 (0x7f431bfc5910) [ CSSD]2013-05-23 17:11:29.171 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5910 (RPC#210) state 6, flags 0x202 [ CSSD]2013-05-23 17:11:29.171 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5910, ret 0, client (nil) [ CSSD]2013-05-23 17:11:29.172 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 210 [ CSSD]2013-05-23 17:11:29.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:29.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:29.717 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:30.889 [1191196992] >TRACE: clssgmAllocProc: (0x2312880) allocated [ CSSD]2013-05-23 17:11:30.890 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:11:30.890 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2312880 - 1,2,3 [ CSSD]2013-05-23 17:11:30.890 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302f30) proc(0x2312880) pid(15196) version 11:1:1:4 [ CSSD]2013-05-23 17:11:30.896 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2312880), client(1/0x22fdc30) [ CSSD]2013-05-23 17:11:30.897 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fdc30) [ CSSD]2013-05-23 17:11:30.897 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fdc30 with con 0x22d7820, requested num -1 [ CSSD]2013-05-23 17:11:30.897 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:30.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 211 (0x7f431bfc5990) [ CSSD]2013-05-23 17:11:30.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5990 (RPC#211) tag(d3002d) sent to node 1 [ CSSD]2013-05-23 17:11:30.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00d3002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:30.897 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015070, gin 1, member 0 [ CSSD]2013-05-23 17:11:30.897 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:11:30.897 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:11:30.897 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:11:30.897 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5990 (RPC#211) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:30.898 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5990, ret 0, client 0x22fdc30 member 0x22e01e0 [ CSSD]2013-05-23 17:11:30.898 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fdc30 [ CSSD]2013-05-23 17:11:30.898 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 211 [ CSSD]2013-05-23 17:11:30.904 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fdc30) [ CSSD]2013-05-23 17:11:30.904 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fdc30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:30.904 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e01e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:30.904 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 212 (0x7f431bfc5a10) [ CSSD]2013-05-23 17:11:30.904 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5a10 (RPC#212) tag(d4002d) sent to node 1 [ CSSD]2013-05-23 17:11:30.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00d4002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:30.904 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:30.904 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5a10 (RPC#212) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:30.904 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5a10, ret 0, client 0x22fdc30 member 0x22e01e0 [ CSSD]2013-05-23 17:11:30.905 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 212 [ CSSD]2013-05-23 17:11:30.905 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e01e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:11:31.107 [1191196992] >TRACE: clssgmDeadProc: proc 0x2312880 [ CSSD]2013-05-23 17:11:31.107 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2312880) con(0x2302f30) skgpid ospid 15196 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:31.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:31.693 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:31.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:32.461 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:32.461 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:32.464 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:11:32.465 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:11:32.465 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:11:32.465 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c121690) proc(0x7f431c12b920) pid(15249) version 11:1:1:4 [ CSSD]2013-05-23 17:11:32.472 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b920), client(1/0x2302f30) [ CSSD]2013-05-23 17:11:32.472 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2302f30) [ CSSD]2013-05-23 17:11:32.472 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2302f30 with con 0x22dfd50, requested num -1 [ CSSD]2013-05-23 17:11:32.472 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:32.472 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 213 (0x7f431bfc5a90) [ CSSD]2013-05-23 17:11:32.472 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5a90 (RPC#213) tag(d5002d) sent to node 1 [ CSSD]2013-05-23 17:11:32.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00d5002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:32.472 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015071, gin 1, member 0 [ CSSD]2013-05-23 17:11:32.472 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:11:32.473 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:11:32.473 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:11:32.473 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5a90 (RPC#213) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:32.473 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5a90, ret 0, client 0x2302f30 member 0x2302d20 [ CSSD]2013-05-23 17:11:32.473 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2302f30 [ CSSD]2013-05-23 17:11:32.473 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 213 [ CSSD]2013-05-23 17:11:32.480 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2302f30) [ CSSD]2013-05-23 17:11:32.480 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2302f30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:32.480 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302d20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:32.480 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 214 (0x7f431bfc5b10) [ CSSD]2013-05-23 17:11:32.480 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5b10 (RPC#214) tag(d6002d) sent to node 1 [ CSSD]2013-05-23 17:11:32.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00d6002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:32.480 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:32.480 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5b10 (RPC#214) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:32.480 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5b10, ret 0, client 0x2302f30 member 0x2302d20 [ CSSD]2013-05-23 17:11:32.480 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 214 [ CSSD]2013-05-23 17:11:32.480 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302d20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:11:32.683 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:11:32.683 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c121690) skgpid ospid 15249 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:33.699 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:33.699 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:33.735 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:35.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:35.730 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:35.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:37.464 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:37.464 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:37.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:37.736 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:37.748 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:39.190 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:11:39.191 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 215 (0x7f431bfc5b90) [ CSSD]2013-05-23 17:11:39.191 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5b90 (RPC#215) tag(d7002d) sent to node 1 [ CSSD]2013-05-23 17:11:39.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:39.191 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 215 (0x7f431bfc5b90) [ CSSD]2013-05-23 17:11:39.191 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5b90 (RPC#215) state 6, flags 0x202 [ CSSD]2013-05-23 17:11:39.191 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5b90, ret 0, client (nil) [ CSSD]2013-05-23 17:11:39.192 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 215 [ CSSD]2013-05-23 17:11:39.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:39.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:39.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:40.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:40.920 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015072, gin 1, member 0 [ CSSD]2013-05-23 17:11:40.920 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:11:40.920 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:11:40.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:40.927 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:11:40.927 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:11:41.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:41.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:41.845 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:42.468 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:42.468 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:43.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:43.819 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:43.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:45.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:45.830 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:45.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:46.148 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:11:46.149 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:11:46.150 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:11:46.150 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302f30) proc(0x22fdb60) pid(15361) version 11:1:1:4 [ CSSD]2013-05-23 17:11:46.156 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x2312950) [ CSSD]2013-05-23 17:11:46.156 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:11:46.156 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 17:11:46.156 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:46.156 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 216 (0x7f431bfc5c10) [ CSSD]2013-05-23 17:11:46.156 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5c10 (RPC#216) tag(d8002d) sent to node 1 [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00d8002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015073, gin 1, member 0 [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5c10 (RPC#216) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5c10, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:11:46.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 216 [ CSSD]2013-05-23 17:11:46.163 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:11:46.163 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:46.163 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:11:46.163 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 217 (0x7f431bfc5c90) [ CSSD]2013-05-23 17:11:46.163 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5c90 (RPC#217) tag(d9002d) sent to node 1 [ CSSD]2013-05-23 17:11:46.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00d9002d) incarnation(174631051) [ CSSD]2013-05-23 17:11:46.164 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:11:46.164 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5c90 (RPC#217) state 6, flags 0x100 [ CSSD]2013-05-23 17:11:46.164 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5c90, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:11:46.164 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 217 [ CSSD]2013-05-23 17:11:46.164 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:11:46.366 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:11:46.366 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x2302f30) skgpid ospid 15361 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:11:46.471 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:46.471 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:11:47.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:47.836 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:47.912 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:49.220 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:11:49.220 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 218 (0x7f431bfc5d10) [ CSSD]2013-05-23 17:11:49.221 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5d10 (RPC#218) tag(da002d) sent to node 1 [ CSSD]2013-05-23 17:11:49.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:49.221 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 218 (0x7f431bfc5d10) [ CSSD]2013-05-23 17:11:49.221 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5d10 (RPC#218) state 6, flags 0x202 [ CSSD]2013-05-23 17:11:49.221 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5d10, ret 0, client (nil) [ CSSD]2013-05-23 17:11:49.221 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 218 [ CSSD]2013-05-23 17:11:49.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:49.889 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:49.919 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:51.475 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:51.475 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:51.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:51.895 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:51.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:53.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:53.901 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:53.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:55.930 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:55.930 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:55.942 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:56.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:56.171 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015074, gin 1, member 0 [ CSSD]2013-05-23 17:11:56.171 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:11:56.171 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:11:56.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:56.211 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:11:56.211 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:11:56.479 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:11:56.479 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:11:57.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:57.935 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:11:57.953 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:59.250 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:11:59.250 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 219 (0x7f431bfc5d90) [ CSSD]2013-05-23 17:11:59.250 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5d90 (RPC#219) tag(db002d) sent to node 1 [ CSSD]2013-05-23 17:11:59.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:59.251 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 219 (0x7f431bfc5d90) [ CSSD]2013-05-23 17:11:59.251 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5d90 (RPC#219) state 6, flags 0x202 [ CSSD]2013-05-23 17:11:59.251 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc5d90, ret 0, client (nil) [ CSSD]2013-05-23 17:11:59.251 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 219 [ CSSD]2013-05-23 17:11:59.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:11:59.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:11:59.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:00.482 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:00.482 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:12:01.409 [1191196992] >TRACE: clssgmAllocProc: (0x2312880) allocated [ CSSD]2013-05-23 17:12:01.410 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:12:01.411 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2312880 - 1,2,3 [ CSSD]2013-05-23 17:12:01.411 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e2f0) proc(0x2312880) pid(15507) version 11:1:1:4 [ CSSD]2013-05-23 17:12:01.427 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2312880), client(1/0x22fdc30) [ CSSD]2013-05-23 17:12:01.427 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22fdc30) [ CSSD]2013-05-23 17:12:01.427 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22fdc30 with con 0x22e01e0, requested num -1 [ CSSD]2013-05-23 17:12:01.427 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:01.427 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 220 (0x7f431bfc5e10) [ CSSD]2013-05-23 17:12:01.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5e10 (RPC#220) tag(dc002d) sent to node 1 [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00dc002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015075, gin 1, member 0 [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5e10 (RPC#220) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5e10, ret 0, client 0x22fdc30 member 0x22d7750 [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22fdc30 [ CSSD]2013-05-23 17:12:01.428 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 220 [ CSSD]2013-05-23 17:12:01.435 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22fdc30) [ CSSD]2013-05-23 17:12:01.435 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22fdc30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:01.435 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:01.435 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 221 (0x7f431bfc5e90) [ CSSD]2013-05-23 17:12:01.435 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5e90 (RPC#221) tag(dd002d) sent to node 1 [ CSSD]2013-05-23 17:12:01.435 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00dd002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:01.435 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:01.435 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5e90 (RPC#221) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:01.435 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5e90, ret 0, client 0x22fdc30 member 0x22d7750 [ CSSD]2013-05-23 17:12:01.435 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 221 [ CSSD]2013-05-23 17:12:01.435 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:01.638 [1191196992] >TRACE: clssgmDeadProc: proc 0x2312880 [ CSSD]2013-05-23 17:12:01.638 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2312880) con(0x205e2f0) skgpid ospid 15507 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:01.986 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:01.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:01.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:04.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:04.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:04.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:05.485 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:05.485 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:06.022 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:06.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:06.030 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:08.033 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:08.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:08.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:08.425 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:12:08.426 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:12:08.427 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:12:08.427 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302f30) proc(0x22fdb60) pid(15590) version 11:1:1:4 [ CSSD]2013-05-23 17:12:08.444 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x2312950) [ CSSD]2013-05-23 17:12:08.444 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:08.444 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22e2eb0, requested num -1 [ CSSD]2013-05-23 17:12:08.444 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:08.444 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 222 (0x7f431bfc5f10) [ CSSD]2013-05-23 17:12:08.444 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5f10 (RPC#222) tag(de002d) sent to node 1 [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00de002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015076, gin 1, member 0 [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5f10 (RPC#222) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc5f10, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:12:08.445 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 222 [ CSSD]2013-05-23 17:12:08.452 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:08.452 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:08.452 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:08.452 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 223 (0x7f431bfc5f90) [ CSSD]2013-05-23 17:12:08.452 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc5f90 (RPC#223) tag(df002d) sent to node 1 [ CSSD]2013-05-23 17:12:08.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00df002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:08.452 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:08.452 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc5f90 (RPC#223) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:08.452 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc5f90, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:08.452 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 223 [ CSSD]2013-05-23 17:12:08.452 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:08.655 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:12:08.655 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x2302f30) skgpid ospid 15590 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:09.280 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:12:09.281 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 224 (0x7f431bfc6010) [ CSSD]2013-05-23 17:12:09.281 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6010 (RPC#224) tag(e0002d) sent to node 1 [ CSSD]2013-05-23 17:12:09.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:09.281 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 224 (0x7f431bfc6010) [ CSSD]2013-05-23 17:12:09.281 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6010 (RPC#224) state 6, flags 0x202 [ CSSD]2013-05-23 17:12:09.281 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6010, ret 0, client (nil) [ CSSD]2013-05-23 17:12:09.281 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 224 [ CSSD]2013-05-23 17:12:10.040 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:10.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:10.090 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:10.489 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:10.489 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:11.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:11.429 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015077, gin 1, member 0 [ CSSD]2013-05-23 17:12:11.429 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:12:11.429 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:12:11.435 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:11.435 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:12:11.435 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:12:12.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:12.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:12.096 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:14.066 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:14.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:14.101 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:15.492 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:15.493 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:16.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:16.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:16.112 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:16.672 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:12:16.673 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:12:16.674 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:12:16.674 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e2f0) proc(0x22fdb60) pid(15880) version 11:1:1:4 [ CSSD]2013-05-23 17:12:16.680 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x2312950) [ CSSD]2013-05-23 17:12:16.680 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:16.680 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22e01e0, requested num -1 [ CSSD]2013-05-23 17:12:16.680 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:16.680 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 225 (0x7f431bfc6090) [ CSSD]2013-05-23 17:12:16.680 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6090 (RPC#225) tag(e1002d) sent to node 1 [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00e1002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015078, gin 1, member 0 [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6090 (RPC#225) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6090, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:16.680 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:12:16.681 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 225 [ CSSD]2013-05-23 17:12:16.687 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:16.687 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:16.687 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:16.687 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 226 (0x7f431bfc6110) [ CSSD]2013-05-23 17:12:16.687 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6110 (RPC#226) tag(e2002d) sent to node 1 [ CSSD]2013-05-23 17:12:16.688 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00e2002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:16.688 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:16.688 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6110 (RPC#226) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:16.688 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6110, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:16.688 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 226 [ CSSD]2013-05-23 17:12:16.688 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:16.890 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:12:16.890 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x205e2f0) skgpid ospid 15880 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:18.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:18.117 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:18.124 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:18.935 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:12:18.936 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:12:18.936 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:12:18.936 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e2f0) proc(0x22fdb60) pid(15933) version 11:1:1:4 [ CSSD]2013-05-23 17:12:19.062 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 17:12:19.063 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:12:19.064 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 17:12:19.064 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7820) proc(0x22d7cd0) pid(15943) version 11:1:1:4 [ CSSD]2013-05-23 17:12:19.082 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:12:19.084 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:12:19.087 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 17:12:19.087 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x22d7820) skgpid ospid 15943 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:19.158 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(77/0x2312950) [ CSSD]2013-05-23 17:12:19.158 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 77 (0x2312950) [ CSSD]2013-05-23 17:12:19.158 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x2312950 with con 0x22d7820, requested num -1 [ CSSD]2013-05-23 17:12:19.158 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:12:19.158 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 227 (0x7f431bfc6190) [ CSSD]2013-05-23 17:12:19.158 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6190 (RPC#227) tag(e3002d) sent to node 1 [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00e3002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015079, gin 1, member 0 [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x22f75a0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6190 (RPC#227) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6190, ret 0, client 0x2312950 member 0x22e2eb0 [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:12:19.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 227 [ CSSD]2013-05-23 17:12:19.166 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 77 (0x2312950) [ CSSD]2013-05-23 17:12:19.166 [1191196992] >TRACE: clssgmExitGrock: client 77 (0x2312950), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:12:19.166 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e2eb0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:12:19.166 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 228 (0x7f431bfc6210) [ CSSD]2013-05-23 17:12:19.166 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6210 (RPC#228) tag(e4002d) sent to node 1 [ CSSD]2013-05-23 17:12:19.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00e4002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:19.166 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:12:19.166 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6210 (RPC#228) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:19.166 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6210, ret 0, client 0x2312950 member 0x22e2eb0 [ CSSD]2013-05-23 17:12:19.166 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 228 [ CSSD]2013-05-23 17:12:19.166 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22e2eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:19.167 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:12:19.167 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x205e2f0) skgpid ospid 15933 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:19.310 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:12:19.310 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 229 (0x7f431bfc6290) [ CSSD]2013-05-23 17:12:19.310 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6290 (RPC#229) tag(e5002d) sent to node 1 [ CSSD]2013-05-23 17:12:19.311 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:19.311 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 229 (0x7f431bfc6290) [ CSSD]2013-05-23 17:12:19.311 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6290 (RPC#229) state 6, flags 0x202 [ CSSD]2013-05-23 17:12:19.311 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6290, ret 0, client (nil) [ CSSD]2013-05-23 17:12:19.311 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 229 [ CSSD]2013-05-23 17:12:19.495 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:19.496 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:12:20.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:20.129 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:20.130 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:22.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:22.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:22.137 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:24.141 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:24.141 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:24.147 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:24.499 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:24.500 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:26.154 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:26.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:26.193 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:26.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:26.682 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015080, gin 1, member 0 [ CSSD]2013-05-23 17:12:26.682 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:12:26.682 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:12:26.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:26.689 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:12:26.689 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:12:28.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:28.198 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:28.244 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:28.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:28.891 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015081, gin 1, member 0 [ CSSD]2013-05-23 17:12:28.891 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:12:28.891 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:12:28.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:28.898 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:12:28.898 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:12:29.330 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:12:29.330 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 230 (0x7f431bfc6310) [ CSSD]2013-05-23 17:12:29.330 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6310 (RPC#230) tag(e6002d) sent to node 1 [ CSSD]2013-05-23 17:12:29.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:29.331 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 230 (0x7f431bfc6310) [ CSSD]2013-05-23 17:12:29.331 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6310 (RPC#230) state 6, flags 0x202 [ CSSD]2013-05-23 17:12:29.331 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6310, ret 0, client (nil) [ CSSD]2013-05-23 17:12:29.331 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 230 [ CSSD]2013-05-23 17:12:29.503 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:29.503 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:30.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:30.209 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:30.251 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:31.923 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:12:31.924 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:12:31.925 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:12:31.925 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e2f0) proc(0x22fdb60) pid(16043) version 11:1:1:4 [ CSSD]2013-05-23 17:12:31.931 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x2312950) [ CSSD]2013-05-23 17:12:31.931 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:31.931 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22e02b0, requested num -1 [ CSSD]2013-05-23 17:12:31.931 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:31.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 231 (0x7f431bfc6390) [ CSSD]2013-05-23 17:12:31.931 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6390 (RPC#231) tag(e7002d) sent to node 1 [ CSSD]2013-05-23 17:12:31.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00e7002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:31.931 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015082, gin 1, member 0 [ CSSD]2013-05-23 17:12:31.931 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:12:31.932 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:31.932 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:12:31.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6390 (RPC#231) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:31.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6390, ret 0, client 0x2312950 member 0x22e2eb0 [ CSSD]2013-05-23 17:12:31.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:12:31.932 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 231 [ CSSD]2013-05-23 17:12:31.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:31.939 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:31.939 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e2eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:31.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 232 (0x7f431bfc6410) [ CSSD]2013-05-23 17:12:31.940 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6410 (RPC#232) tag(e8002d) sent to node 1 [ CSSD]2013-05-23 17:12:31.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00e8002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:31.940 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:31.940 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6410 (RPC#232) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:31.940 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6410, ret 0, client 0x2312950 member 0x22e2eb0 [ CSSD]2013-05-23 17:12:31.940 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 232 [ CSSD]2013-05-23 17:12:31.940 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e2eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:32.143 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:12:32.143 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x205e2f0) skgpid ospid 16043 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:32.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:32.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:32.257 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:32.726 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:12:32.727 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:12:32.728 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:12:32.728 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302f30) proc(0x22fdb60) pid(16091) version 11:1:1:4 [ CSSD]2013-05-23 17:12:32.745 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x2312950) [ CSSD]2013-05-23 17:12:32.745 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:32.745 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 17:12:32.745 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:32.745 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 233 (0x7f431bfc6490) [ CSSD]2013-05-23 17:12:32.745 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6490 (RPC#233) tag(e9002d) sent to node 1 [ CSSD]2013-05-23 17:12:32.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00e9002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:32.745 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015083, gin 1, member 0 [ CSSD]2013-05-23 17:12:32.745 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:12:32.745 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:32.746 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:12:32.746 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6490 (RPC#233) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:32.746 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6490, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:32.746 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:12:32.746 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 233 [ CSSD]2013-05-23 17:12:32.751 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:32.751 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:32.751 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:32.751 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 234 (0x7f431bfc6510) [ CSSD]2013-05-23 17:12:32.752 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6510 (RPC#234) tag(ea002d) sent to node 1 [ CSSD]2013-05-23 17:12:32.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00ea002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:32.752 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:32.752 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6510 (RPC#234) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:32.752 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6510, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:32.752 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 234 [ CSSD]2013-05-23 17:12:32.752 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:32.955 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:12:32.955 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x2302f30) skgpid ospid 16091 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:34.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:34.240 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:34.267 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:34.507 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:34.507 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:36.273 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:36.313 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:36.313 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:38.319 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:38.319 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:38.325 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:38.510 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:38.510 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:12:39.360 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:12:39.361 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 235 (0x7f431bfc6590) [ CSSD]2013-05-23 17:12:39.361 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6590 (RPC#235) tag(eb002d) sent to node 1 [ CSSD]2013-05-23 17:12:39.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:39.361 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 235 (0x7f431bfc6590) [ CSSD]2013-05-23 17:12:39.361 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6590 (RPC#235) state 6, flags 0x202 [ CSSD]2013-05-23 17:12:39.361 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6590, ret 0, client (nil) [ CSSD]2013-05-23 17:12:39.362 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 235 [ CSSD]2013-05-23 17:12:40.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:40.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:40.332 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:41.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:41.942 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015084, gin 1, member 0 [ CSSD]2013-05-23 17:12:41.942 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:12:41.942 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:12:41.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:41.981 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:12:41.981 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:12:42.337 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:42.337 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:42.338 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:43.514 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:43.514 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:44.348 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:44.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:44.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:46.355 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:46.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:46.364 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:47.181 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:12:47.183 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:12:47.183 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:12:47.183 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e2f0) proc(0x22fdb60) pid(16269) version 11:1:1:4 [ CSSD]2013-05-23 17:12:47.190 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x2312950) [ CSSD]2013-05-23 17:12:47.190 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:47.190 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22e01e0, requested num -1 [ CSSD]2013-05-23 17:12:47.190 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:47.190 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 236 (0x7f431bfc6610) [ CSSD]2013-05-23 17:12:47.190 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6610 (RPC#236) tag(ec002d) sent to node 1 [ CSSD]2013-05-23 17:12:47.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00ec002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:47.190 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015085, gin 1, member 0 [ CSSD]2013-05-23 17:12:47.190 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f75a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:12:47.191 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:12:47.191 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:12:47.191 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6610 (RPC#236) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:47.191 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6610, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:47.191 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:12:47.191 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 236 [ CSSD]2013-05-23 17:12:47.198 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:12:47.198 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:47.198 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7750) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:12:47.198 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 237 (0x7f431bfc6690) [ CSSD]2013-05-23 17:12:47.198 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6690 (RPC#237) tag(ed002d) sent to node 1 [ CSSD]2013-05-23 17:12:47.198 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00ed002d) incarnation(174631051) [ CSSD]2013-05-23 17:12:47.198 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:12:47.198 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6690 (RPC#237) state 6, flags 0x100 [ CSSD]2013-05-23 17:12:47.198 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6690, ret 0, client 0x2312950 member 0x22d7750 [ CSSD]2013-05-23 17:12:47.198 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 237 [ CSSD]2013-05-23 17:12:47.198 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7750) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:12:47.401 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:12:47.401 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x205e2f0) skgpid ospid 16269 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:48.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:48.370 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:48.370 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:48.518 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:48.518 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:49.381 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:12:49.381 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 238 (0x7f431bfc6710) [ CSSD]2013-05-23 17:12:49.382 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6710 (RPC#238) tag(ee002d) sent to node 1 [ CSSD]2013-05-23 17:12:49.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:49.382 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 238 (0x7f431bfc6710) [ CSSD]2013-05-23 17:12:49.382 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6710 (RPC#238) state 6, flags 0x202 [ CSSD]2013-05-23 17:12:49.382 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6710, ret 0, client (nil) [ CSSD]2013-05-23 17:12:49.382 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 238 [ CSSD]2013-05-23 17:12:50.000 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22dffb0) [ CSSD]2013-05-23 17:12:50.000 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22dffb0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:12:50.000 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22dffb0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:12:50.000 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:12:50.001 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed20 [ CSSD]2013-05-23 17:12:50.001 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed20) con(0x205e240) skgpid 10708 ospid 10708 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:12:50.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:50.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:50.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:52.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:52.388 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:52.398 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:52.520 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:52.521 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:12:54.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:54.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:54.429 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:56.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:56.415 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:56.435 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:57.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:57.202 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015086, gin 1, member 0 [ CSSD]2013-05-23 17:12:57.202 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:12:57.202 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:12:57.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:57.241 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:12:57.241 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:12:57.524 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:12:57.524 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:12:58.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:58.440 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:12:58.526 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:12:59.406 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:12:59.407 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 239 (0x7f431bfc6790) [ CSSD]2013-05-23 17:12:59.407 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6790 (RPC#239) tag(ef002d) sent to node 1 [ CSSD]2013-05-23 17:12:59.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:12:59.407 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 239 (0x7f431bfc6790) [ CSSD]2013-05-23 17:12:59.407 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6790 (RPC#239) state 6, flags 0x202 [ CSSD]2013-05-23 17:12:59.407 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6790, ret 0, client (nil) [ CSSD]2013-05-23 17:12:59.408 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 239 [ CSSD]2013-05-23 17:13:00.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:00.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:00.533 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:02.434 [1191196992] >TRACE: clssgmAllocProc: (0x22e2eb0) allocated [ CSSD]2013-05-23 17:13:02.435 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:13:02.435 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e2eb0 - 1,2,3 [ CSSD]2013-05-23 17:13:02.435 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22e2eb0) pid(16426) version 11:1:1:4 [ CSSD]2013-05-23 17:13:02.442 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e2eb0), client(1/0x2312940) [ CSSD]2013-05-23 17:13:02.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:02.442 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312940 with con 0x22d7820, requested num -1 [ CSSD]2013-05-23 17:13:02.442 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:02.442 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 240 (0x7f431bfc6810) [ CSSD]2013-05-23 17:13:02.442 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6810 (RPC#240) tag(f0002d) sent to node 1 [ CSSD]2013-05-23 17:13:02.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00f0002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:02.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015087, gin 1, member 0 [ CSSD]2013-05-23 17:13:02.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:13:02.442 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:13:02.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:13:02.443 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6810 (RPC#240) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:02.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6810, ret 0, client 0x2312940 member 0x22fdb60 [ CSSD]2013-05-23 17:13:02.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312940 [ CSSD]2013-05-23 17:13:02.443 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 240 [ CSSD]2013-05-23 17:13:02.450 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:02.450 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:02.450 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:02.450 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 241 (0x7f431bfc6890) [ CSSD]2013-05-23 17:13:02.450 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6890 (RPC#241) tag(f1002d) sent to node 1 [ CSSD]2013-05-23 17:13:02.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00f1002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:02.451 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:02.451 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6890 (RPC#241) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:02.451 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6890, ret 0, client 0x2312940 member 0x22fdb60 [ CSSD]2013-05-23 17:13:02.451 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 241 [ CSSD]2013-05-23 17:13:02.451 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:13:02.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:02.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:02.528 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:02.528 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:02.540 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:02.653 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e2eb0 [ CSSD]2013-05-23 17:13:02.653 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e2eb0) con(0x2317510) skgpid ospid 16426 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:13:04.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:04.510 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:04.549 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:06.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:06.516 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:06.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:07.531 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:07.531 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:08.542 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:08.542 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:08.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:09.430 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:13:09.430 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 242 (0x7f431bfc6910) [ CSSD]2013-05-23 17:13:09.430 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6910 (RPC#242) tag(f2002d) sent to node 1 [ CSSD]2013-05-23 17:13:09.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:09.431 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 242 (0x7f431bfc6910) [ CSSD]2013-05-23 17:13:09.431 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6910 (RPC#242) state 6, flags 0x202 [ CSSD]2013-05-23 17:13:09.431 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6910, ret 0, client (nil) [ CSSD]2013-05-23 17:13:09.431 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 242 [ CSSD]2013-05-23 17:13:10.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:10.552 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:10.572 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:11.534 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:11.534 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:13:12.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:12.453 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015088, gin 1, member 0 [ CSSD]2013-05-23 17:13:12.453 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:13:12.454 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:13:12.494 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:12.494 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:13:12.494 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:13:12.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:12.558 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:12.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:14.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:14.610 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:14.630 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:16.537 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:16.538 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:16.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:16.616 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:16.637 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:17.689 [1191196992] >TRACE: clssgmAllocProc: (0x22e2eb0) allocated [ CSSD]2013-05-23 17:13:17.690 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:13:17.691 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e2eb0 - 1,2,3 [ CSSD]2013-05-23 17:13:17.691 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d20) proc(0x22e2eb0) pid(16582) version 11:1:1:4 [ CSSD]2013-05-23 17:13:17.697 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e2eb0), client(1/0x2312940) [ CSSD]2013-05-23 17:13:17.697 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:17.697 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312940 with con 0x22d7820, requested num -1 [ CSSD]2013-05-23 17:13:17.697 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:17.697 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 243 (0x7f431bfc6990) [ CSSD]2013-05-23 17:13:17.697 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6990 (RPC#243) tag(f3002d) sent to node 1 [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00f3002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015089, gin 1, member 0 [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6990 (RPC#243) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6990, ret 0, client 0x2312940 member 0x2302b40 [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312940 [ CSSD]2013-05-23 17:13:17.698 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 243 [ CSSD]2013-05-23 17:13:17.704 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:17.704 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:17.704 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302b40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:17.704 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 244 (0x7f431bfc6a10) [ CSSD]2013-05-23 17:13:17.704 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6a10 (RPC#244) tag(f4002d) sent to node 1 [ CSSD]2013-05-23 17:13:17.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00f4002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:17.704 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:17.704 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6a10 (RPC#244) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:17.704 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6a10, ret 0, client 0x2312940 member 0x2302b40 [ CSSD]2013-05-23 17:13:17.704 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 244 [ CSSD]2013-05-23 17:13:17.704 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302b40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:13:17.907 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e2eb0 [ CSSD]2013-05-23 17:13:17.907 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e2eb0) con(0x2302d20) skgpid ospid 16582 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:13:18.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:18.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:18.727 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:19.460 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:13:19.460 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 245 (0x7f431bfc6a90) [ CSSD]2013-05-23 17:13:19.460 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6a90 (RPC#245) tag(f5002d) sent to node 1 [ CSSD]2013-05-23 17:13:19.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:19.461 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 245 (0x7f431bfc6a90) [ CSSD]2013-05-23 17:13:19.461 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6a90 (RPC#245) state 6, flags 0x202 [ CSSD]2013-05-23 17:13:19.461 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6a90, ret 0, client (nil) [ CSSD]2013-05-23 17:13:19.461 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 245 [ CSSD]2013-05-23 17:13:20.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:20.653 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:20.752 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:21.541 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:21.542 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:22.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:22.659 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:22.758 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:24.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:24.711 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:24.771 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:26.545 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:26.545 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:26.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:26.717 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:26.777 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:27.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:27.703 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015090, gin 1, member 0 [ CSSD]2013-05-23 17:13:27.703 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:13:27.703 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:13:27.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:27.710 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:13:27.710 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:13:28.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:28.722 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:28.777 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:29.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:29.145 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015091, gin 1, member 0 [ CSSD]2013-05-23 17:13:29.145 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:13:29.145 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:13:29.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:29.152 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:13:29.152 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:13:29.484 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:13:29.485 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 246 (0x7f431bfc6b10) [ CSSD]2013-05-23 17:13:29.485 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6b10 (RPC#246) tag(f6002d) sent to node 1 [ CSSD]2013-05-23 17:13:29.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:29.485 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 246 (0x7f431bfc6b10) [ CSSD]2013-05-23 17:13:29.485 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6b10 (RPC#246) state 6, flags 0x202 [ CSSD]2013-05-23 17:13:29.485 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6b10, ret 0, client (nil) [ CSSD]2013-05-23 17:13:29.486 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 246 [ CSSD]2013-05-23 17:13:30.547 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:30.548 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:13:30.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:30.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:30.784 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:32.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:32.760 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:32.791 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:32.945 [1191196992] >TRACE: clssgmAllocProc: (0x22e2eb0) allocated [ CSSD]2013-05-23 17:13:32.946 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:13:32.946 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e2eb0 - 1,2,3 [ CSSD]2013-05-23 17:13:32.946 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x22e2eb0) pid(16723) version 11:1:1:4 [ CSSD]2013-05-23 17:13:32.952 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e2eb0), client(1/0x2312940) [ CSSD]2013-05-23 17:13:32.952 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:32.952 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312940 with con 0x22d7820, requested num -1 [ CSSD]2013-05-23 17:13:32.952 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:32.952 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 247 (0x7f431bfc6b90) [ CSSD]2013-05-23 17:13:32.952 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6b90 (RPC#247) tag(f7002d) sent to node 1 [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00f7002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015092, gin 1, member 0 [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6b90 (RPC#247) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6b90, ret 0, client 0x2312940 member 0x231d6c0 [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312940 [ CSSD]2013-05-23 17:13:32.953 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 247 [ CSSD]2013-05-23 17:13:32.959 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:32.959 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:32.959 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231d6c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:32.959 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 248 (0x7f431bfc6c10) [ CSSD]2013-05-23 17:13:32.960 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6c10 (RPC#248) tag(f8002d) sent to node 1 [ CSSD]2013-05-23 17:13:32.960 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00f8002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:32.960 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:32.960 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6c10 (RPC#248) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:32.960 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6c10, ret 0, client 0x2312940 member 0x231d6c0 [ CSSD]2013-05-23 17:13:32.960 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 248 [ CSSD]2013-05-23 17:13:32.960 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231d6c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:13:32.999 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:13:33.000 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:13:33.000 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:13:33.000 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(16729) version 11:1:1:4 [ CSSD]2013-05-23 17:13:33.006 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x2312940) [ CSSD]2013-05-23 17:13:33.007 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:33.007 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312940 with con 0x22d7820, requested num -1 [ CSSD]2013-05-23 17:13:33.007 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:33.007 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 249 (0x7f431bfc6c90) [ CSSD]2013-05-23 17:13:33.007 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6c90 (RPC#249) tag(f9002d) sent to node 1 [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00f9002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015093, gin 1, member 0 [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6c90 (RPC#249) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6c90, ret 0, client 0x2312940 member 0x2303710 [ CSSD]2013-05-23 17:13:33.007 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312940 [ CSSD]2013-05-23 17:13:33.008 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 249 [ CSSD]2013-05-23 17:13:33.014 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:33.014 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:33.014 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:33.014 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 250 (0x7f431bfc6d10) [ CSSD]2013-05-23 17:13:33.014 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6d10 (RPC#250) tag(fa002d) sent to node 1 [ CSSD]2013-05-23 17:13:33.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00fa002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:33.015 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:33.015 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6d10 (RPC#250) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:33.015 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6d10, ret 0, client 0x2312940 member 0x2303710 [ CSSD]2013-05-23 17:13:33.015 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 250 [ CSSD]2013-05-23 17:13:33.015 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:13:33.162 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e2eb0 [ CSSD]2013-05-23 17:13:33.162 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e2eb0) con(0x2317510) skgpid ospid 16723 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:13:33.217 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:13:33.217 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 16729 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:13:34.811 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:34.811 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:34.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:35.314 [1191196992] >TRACE: clssgmAllocProc: (0x22e2eb0) allocated [ CSSD]2013-05-23 17:13:35.315 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:13:35.315 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e2eb0 - 1,2,3 [ CSSD]2013-05-23 17:13:35.315 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2302d20) proc(0x22e2eb0) pid(16338/16338) version 11:1:1:4 [ CSSD]2013-05-23 17:13:35.322 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22e2eb0), client(1/0x2312940) [ CSSD]2013-05-23 17:13:35.322 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:13:35.322 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:13:35.322 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:13:35.551 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:35.552 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:36.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:36.817 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:36.848 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:38.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:38.823 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:38.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:39.521 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:13:39.522 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 251 (0x7f431bfc6d90) [ CSSD]2013-05-23 17:13:39.522 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6d90 (RPC#251) tag(fb002d) sent to node 1 [ CSSD]2013-05-23 17:13:39.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:39.522 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 251 (0x7f431bfc6d90) [ CSSD]2013-05-23 17:13:39.522 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6d90 (RPC#251) state 6, flags 0x202 [ CSSD]2013-05-23 17:13:39.522 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6d90, ret 0, client (nil) [ CSSD]2013-05-23 17:13:39.523 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 251 [ CSSD]2013-05-23 17:13:40.555 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:40.555 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:40.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:40.834 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:40.866 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:41.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:41.656 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015094, gin 1, member 0 [ CSSD]2013-05-23 17:13:41.656 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:13:41.656 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:13:41.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:41.696 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:13:41.696 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:13:41.787 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:13:41.788 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:13:41.789 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:13:41.789 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:13:41.791 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:13:41.791 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:13:42.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:42.859 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:42.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:42.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:42.956 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015095, gin 1, member 0 [ CSSD]2013-05-23 17:13:42.956 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:13:42.956 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:13:42.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:42.995 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:13:42.995 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:13:44.558 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:44.558 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:13:44.882 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:44.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:44.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:46.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:46.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:46.917 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:48.194 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:13:48.195 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:13:48.196 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:13:48.196 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(16873) version 11:1:1:4 [ CSSD]2013-05-23 17:13:48.202 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22ff870) [ CSSD]2013-05-23 17:13:48.202 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:13:48.202 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x23037e0, requested num -1 [ CSSD]2013-05-23 17:13:48.202 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:48.202 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 252 (0x7f431bfc6e10) [ CSSD]2013-05-23 17:13:48.202 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6e10 (RPC#252) tag(fc002d) sent to node 1 [ CSSD]2013-05-23 17:13:48.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00fc002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:48.202 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015096, gin 1, member 0 [ CSSD]2013-05-23 17:13:48.202 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:13:48.203 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:13:48.203 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:13:48.203 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6e10 (RPC#252) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:48.203 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc6e10, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:13:48.203 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:13:48.203 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 252 [ CSSD]2013-05-23 17:13:48.209 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:13:48.209 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:48.209 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:13:48.209 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 253 (0x7f431bfc6e90) [ CSSD]2013-05-23 17:13:48.209 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6e90 (RPC#253) tag(fd002d) sent to node 1 [ CSSD]2013-05-23 17:13:48.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00fd002d) incarnation(174631051) [ CSSD]2013-05-23 17:13:48.210 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:13:48.210 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6e90 (RPC#253) state 6, flags 0x100 [ CSSD]2013-05-23 17:13:48.210 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc6e90, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:13:48.210 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 253 [ CSSD]2013-05-23 17:13:48.210 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:13:48.413 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:13:48.413 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 16873 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:13:48.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:48.922 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:48.939 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:49.546 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:13:49.547 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 254 (0x7f431bfc6f10) [ CSSD]2013-05-23 17:13:49.547 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6f10 (RPC#254) tag(fe002d) sent to node 1 [ CSSD]2013-05-23 17:13:49.547 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:49.547 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 254 (0x7f431bfc6f10) [ CSSD]2013-05-23 17:13:49.547 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6f10 (RPC#254) state 6, flags 0x202 [ CSSD]2013-05-23 17:13:49.547 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6f10, ret 0, client (nil) [ CSSD]2013-05-23 17:13:49.548 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 254 [ CSSD]2013-05-23 17:13:49.562 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:49.562 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:50.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:50.935 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:50.946 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:52.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:52.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:52.980 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:54.566 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:54.566 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:54.963 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:55.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:55.012 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:56.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:57.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:57.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:58.216 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:58.216 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015097, gin 1, member 0 [ CSSD]2013-05-23 17:13:58.216 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:13:58.216 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:13:58.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:58.223 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:13:58.223 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:13:58.978 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:13:59.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:59.024 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:13:59.569 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:13:59.570 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:13:59.580 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:13:59.580 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 255 (0x7f431bfc6f90) [ CSSD]2013-05-23 17:13:59.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc6f90 (RPC#255) tag(ff002d) sent to node 1 [ CSSD]2013-05-23 17:13:59.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:13:59.581 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 255 (0x7f431bfc6f90) [ CSSD]2013-05-23 17:13:59.581 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc6f90 (RPC#255) state 6, flags 0x202 [ CSSD]2013-05-23 17:13:59.581 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc6f90, ret 0, client (nil) [ CSSD]2013-05-23 17:13:59.581 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 255 [ CSSD]2013-05-23 17:14:00.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:01.035 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:01.035 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:03.037 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:03.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:03.060 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:03.449 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:14:03.450 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:14:03.451 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:14:03.451 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(17028) version 11:1:1:4 [ CSSD]2013-05-23 17:14:03.457 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22ff870) [ CSSD]2013-05-23 17:14:03.457 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:03.457 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x23037e0, requested num -1 [ CSSD]2013-05-23 17:14:03.457 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:03.457 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 256 (0x7f431bfc7010) [ CSSD]2013-05-23 17:14:03.458 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7010 (RPC#256) tag(100002d) sent to node 1 [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0100002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015098, gin 1, member 0 [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7010 (RPC#256) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7010, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:14:03.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 256 [ CSSD]2013-05-23 17:14:03.464 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:03.464 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:03.464 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:03.464 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 257 (0x7f431bfc7090) [ CSSD]2013-05-23 17:14:03.464 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7090 (RPC#257) tag(101002d) sent to node 1 [ CSSD]2013-05-23 17:14:03.464 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0101002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:03.464 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:03.464 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7090 (RPC#257) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:03.464 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7090, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:14:03.465 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 257 [ CSSD]2013-05-23 17:14:03.465 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:14:03.572 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:03.573 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:14:03.667 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:14:03.667 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 17028 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:14:05.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:05.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:05.113 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:07.052 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:07.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:07.119 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:08.576 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:08.576 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:09.060 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:09.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:09.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:09.605 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:14:09.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 258 (0x7f431bfc7110) [ CSSD]2013-05-23 17:14:09.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7110 (RPC#258) tag(102002d) sent to node 1 [ CSSD]2013-05-23 17:14:09.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:09.606 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 258 (0x7f431bfc7110) [ CSSD]2013-05-23 17:14:09.606 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7110 (RPC#258) state 6, flags 0x202 [ CSSD]2013-05-23 17:14:09.606 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7110, ret 0, client (nil) [ CSSD]2013-05-23 17:14:09.606 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 258 [ CSSD]2013-05-23 17:14:11.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:11.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:11.145 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:13.077 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:13.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:13.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:13.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:13.467 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015099, gin 1, member 0 [ CSSD]2013-05-23 17:14:13.467 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:14:13.467 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:14:13.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:13.474 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:14:13.474 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:14:13.579 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:13.580 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:15.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:15.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:15.242 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:17.090 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:17.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:17.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:18.583 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:18.584 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:18.700 [1191196992] >TRACE: clssgmAllocProc: (0x2303710) allocated [ CSSD]2013-05-23 17:14:18.701 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:14:18.702 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2303710 - 1,2,3 [ CSSD]2013-05-23 17:14:18.702 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x2303710) pid(17185) version 11:1:1:4 [ CSSD]2013-05-23 17:14:18.708 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2303710), client(1/0x22ff870) [ CSSD]2013-05-23 17:14:18.708 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:18.708 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x22fca90, requested num -1 [ CSSD]2013-05-23 17:14:18.708 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:18.708 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 259 (0x7f431bfc7190) [ CSSD]2013-05-23 17:14:18.708 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7190 (RPC#259) tag(103002d) sent to node 1 [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0103002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015100, gin 1, member 0 [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7190 (RPC#259) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7190, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:14:18.709 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 259 [ CSSD]2013-05-23 17:14:18.716 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:18.716 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:18.716 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:18.716 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 260 (0x7f431bfc7210) [ CSSD]2013-05-23 17:14:18.716 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7210 (RPC#260) tag(104002d) sent to node 1 [ CSSD]2013-05-23 17:14:18.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0104002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:18.717 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:18.717 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7210 (RPC#260) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:18.717 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7210, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:14:18.717 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 260 [ CSSD]2013-05-23 17:14:18.717 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:14:18.919 [1191196992] >TRACE: clssgmDeadProc: proc 0x2303710 [ CSSD]2013-05-23 17:14:18.919 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2303710) con(0x230f280) skgpid ospid 17185 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:14:19.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:19.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:19.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:19.629 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:14:19.629 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 261 (0x7f431bfc7290) [ CSSD]2013-05-23 17:14:19.630 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7290 (RPC#261) tag(105002d) sent to node 1 [ CSSD]2013-05-23 17:14:19.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:19.630 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 261 (0x7f431bfc7290) [ CSSD]2013-05-23 17:14:19.630 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7290 (RPC#261) state 6, flags 0x202 [ CSSD]2013-05-23 17:14:19.630 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7290, ret 0, client (nil) [ CSSD]2013-05-23 17:14:19.630 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 261 [ CSSD]2013-05-23 17:14:21.147 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:21.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:21.266 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:23.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:23.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:23.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:23.587 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:23.587 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:25.163 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:25.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:25.324 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:27.189 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:27.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:27.330 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:28.591 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:28.591 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:28.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:28.717 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015101, gin 1, member 0 [ CSSD]2013-05-23 17:14:28.717 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:14:28.717 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:14:28.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:28.723 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:14:28.723 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:14:28.816 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:14:28.816 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:14:28.817 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:14:28.818 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:14:28.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:28.983 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015102, gin 1, member 0 [ CSSD]2013-05-23 17:14:28.983 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c1215d0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:14:28.983 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:14:28.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:28.990 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:14:28.990 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:14:29.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:29.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:29.336 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:29.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:29.402 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015103, gin 1, member 0 [ CSSD]2013-05-23 17:14:29.402 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:14:29.402 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:14:29.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:29.408 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:14:29.408 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:14:29.656 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:14:29.657 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 262 (0x7f431bfc7310) [ CSSD]2013-05-23 17:14:29.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7310 (RPC#262) tag(106002d) sent to node 1 [ CSSD]2013-05-23 17:14:29.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:29.657 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 262 (0x7f431bfc7310) [ CSSD]2013-05-23 17:14:29.657 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7310 (RPC#262) state 6, flags 0x202 [ CSSD]2013-05-23 17:14:29.657 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7310, ret 0, client (nil) [ CSSD]2013-05-23 17:14:29.658 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 262 [ CSSD]2013-05-23 17:14:31.267 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:31.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:31.347 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:33.262 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:14:33.263 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:14:33.264 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:14:33.264 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(17323) version 11:1:1:4 [ CSSD]2013-05-23 17:14:33.270 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22ff870) [ CSSD]2013-05-23 17:14:33.270 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:33.270 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x23037e0, requested num -1 [ CSSD]2013-05-23 17:14:33.270 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:33.270 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 263 (0x7f431bfc7390) [ CSSD]2013-05-23 17:14:33.271 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7390 (RPC#263) tag(107002d) sent to node 1 [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0107002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015104, gin 1, member 0 [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7390 (RPC#263) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7390, ret 0, client 0x22ff870 member 0x22d7a00 [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:14:33.271 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 263 [ CSSD]2013-05-23 17:14:33.278 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:33.278 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:33.278 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:33.278 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 264 (0x7f431bfc7410) [ CSSD]2013-05-23 17:14:33.278 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7410 (RPC#264) tag(108002d) sent to node 1 [ CSSD]2013-05-23 17:14:33.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0108002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:33.278 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:33.278 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7410 (RPC#264) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:33.278 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7410, ret 0, client 0x22ff870 member 0x22d7a00 [ CSSD]2013-05-23 17:14:33.279 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 264 [ CSSD]2013-05-23 17:14:33.279 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:14:33.359 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:33.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:33.373 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:33.481 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:14:33.481 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 17323 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:14:33.595 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:33.595 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:33.957 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:14:33.958 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:14:33.959 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:14:33.959 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(17338) version 11:1:1:4 [ CSSD]2013-05-23 17:14:33.966 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22ff870) [ CSSD]2013-05-23 17:14:33.966 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:33.966 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x23037e0, requested num -1 [ CSSD]2013-05-23 17:14:33.966 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:33.966 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 265 (0x7f431bfc7490) [ CSSD]2013-05-23 17:14:33.966 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7490 (RPC#265) tag(109002d) sent to node 1 [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0109002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015105, gin 1, member 0 [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7490 (RPC#265) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7490, ret 0, client 0x22ff870 member 0x22d7a00 [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:14:33.967 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 265 [ CSSD]2013-05-23 17:14:33.973 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:33.973 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:33.973 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:33.973 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 266 (0x7f431bfc7510) [ CSSD]2013-05-23 17:14:33.973 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7510 (RPC#266) tag(10a002d) sent to node 1 [ CSSD]2013-05-23 17:14:33.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(010a002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:33.974 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:33.974 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7510 (RPC#266) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:33.974 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7510, ret 0, client 0x22ff870 member 0x22d7a00 [ CSSD]2013-05-23 17:14:33.974 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 266 [ CSSD]2013-05-23 17:14:33.974 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:14:34.176 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:14:34.177 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 17338 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:14:35.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:35.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:35.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:37.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:37.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:37.450 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:37.598 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:37.598 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:14:39.381 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:39.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:39.461 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:39.679 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:14:39.680 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 267 (0x7f431bfc7590) [ CSSD]2013-05-23 17:14:39.680 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7590 (RPC#267) tag(10b002d) sent to node 1 [ CSSD]2013-05-23 17:14:39.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:39.680 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 267 (0x7f431bfc7590) [ CSSD]2013-05-23 17:14:39.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7590 (RPC#267) state 6, flags 0x202 [ CSSD]2013-05-23 17:14:39.680 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7590, ret 0, client (nil) [ CSSD]2013-05-23 17:14:39.681 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 267 [ CSSD]2013-05-23 17:14:41.387 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:41.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:41.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:42.601 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:42.602 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:43.439 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:43.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:43.472 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:43.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:43.982 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015106, gin 1, member 0 [ CSSD]2013-05-23 17:14:43.982 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:14:43.982 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:14:43.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:43.989 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:14:43.989 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:14:45.446 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:45.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:45.525 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:47.453 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:47.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:47.530 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:47.605 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:47.605 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:49.214 [1191196992] >TRACE: clssgmAllocProc: (0x2303710) allocated [ CSSD]2013-05-23 17:14:49.215 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:14:49.216 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2303710 - 1,2,3 [ CSSD]2013-05-23 17:14:49.216 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x2303710) pid(17488) version 11:1:1:4 [ CSSD]2013-05-23 17:14:49.232 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2303710), client(1/0x22ff870) [ CSSD]2013-05-23 17:14:49.232 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:49.232 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 17:14:49.232 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:49.232 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 268 (0x7f431bfc7610) [ CSSD]2013-05-23 17:14:49.232 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7610 (RPC#268) tag(10c002d) sent to node 1 [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(010c002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015107, gin 1, member 0 [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7610 (RPC#268) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7610, ret 0, client 0x22ff870 member 0x22d7a00 [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:14:49.233 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 268 [ CSSD]2013-05-23 17:14:49.239 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:14:49.239 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:49.239 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:14:49.239 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 269 (0x7f431bfc7690) [ CSSD]2013-05-23 17:14:49.239 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7690 (RPC#269) tag(10d002d) sent to node 1 [ CSSD]2013-05-23 17:14:49.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(010d002d) incarnation(174631051) [ CSSD]2013-05-23 17:14:49.239 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:14:49.239 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7690 (RPC#269) state 6, flags 0x100 [ CSSD]2013-05-23 17:14:49.239 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7690, ret 0, client 0x22ff870 member 0x22d7a00 [ CSSD]2013-05-23 17:14:49.239 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 269 [ CSSD]2013-05-23 17:14:49.239 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:14:49.442 [1191196992] >TRACE: clssgmDeadProc: proc 0x2303710 [ CSSD]2013-05-23 17:14:49.442 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2303710) con(0x230f280) skgpid ospid 17488 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:14:49.461 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:49.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:49.543 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:49.703 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:14:49.703 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 270 (0x7f431bfc7710) [ CSSD]2013-05-23 17:14:49.704 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7710 (RPC#270) tag(10e002d) sent to node 1 [ CSSD]2013-05-23 17:14:49.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:49.704 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 270 (0x7f431bfc7710) [ CSSD]2013-05-23 17:14:49.704 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7710 (RPC#270) state 6, flags 0x202 [ CSSD]2013-05-23 17:14:49.704 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7710, ret 0, client (nil) [ CSSD]2013-05-23 17:14:49.704 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 270 [ CSSD]2013-05-23 17:14:51.468 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:51.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:51.549 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:51.608 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:51.608 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:14:53.480 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:53.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:53.554 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:55.487 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:55.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:55.565 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:56.612 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:14:56.612 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:14:57.494 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:57.571 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:57.571 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:59.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:59.232 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015108, gin 1, member 0 [ CSSD]2013-05-23 17:14:59.232 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:14:59.232 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:14:59.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:59.239 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:14:59.239 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:14:59.513 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:14:59.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:59.623 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:14:59.729 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:14:59.729 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 271 (0x7f431bfc7790) [ CSSD]2013-05-23 17:14:59.729 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7790 (RPC#271) tag(10f002d) sent to node 1 [ CSSD]2013-05-23 17:14:59.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:14:59.730 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 271 (0x7f431bfc7790) [ CSSD]2013-05-23 17:14:59.730 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7790 (RPC#271) state 6, flags 0x202 [ CSSD]2013-05-23 17:14:59.730 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7790, ret 0, client (nil) [ CSSD]2013-05-23 17:14:59.730 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 271 [ CSSD]2013-05-23 17:15:01.520 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:01.615 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:01.616 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:01.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:01.630 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:03.550 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:03.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:03.636 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:04.479 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:15:04.480 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:15:04.481 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:15:04.481 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(17631) version 11:1:1:4 [ CSSD]2013-05-23 17:15:04.488 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22ff870) [ CSSD]2013-05-23 17:15:04.488 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:15:04.488 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x23037e0, requested num -1 [ CSSD]2013-05-23 17:15:04.488 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:04.488 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 272 (0x7f431bfc7810) [ CSSD]2013-05-23 17:15:04.488 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7810 (RPC#272) tag(110002d) sent to node 1 [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0110002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015109, gin 1, member 0 [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7810 (RPC#272) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7810, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:15:04.488 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:15:04.489 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 272 [ CSSD]2013-05-23 17:15:04.495 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:15:04.495 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:04.495 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:04.495 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 273 (0x7f431bfc7890) [ CSSD]2013-05-23 17:15:04.495 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7890 (RPC#273) tag(111002d) sent to node 1 [ CSSD]2013-05-23 17:15:04.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0111002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:04.496 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:04.496 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7890 (RPC#273) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:04.496 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7890, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:15:04.496 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 273 [ CSSD]2013-05-23 17:15:04.496 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:15:04.698 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:15:04.698 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 17631 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:05.557 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:05.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:05.645 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:06.619 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:06.619 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:07.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:07.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:07.651 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:08.412 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:15:08.413 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:15:08.414 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:15:08.414 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(17715) version 11:1:1:4 [ CSSD]2013-05-23 17:15:08.982 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:15:08.982 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid ospid 17715 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:09.572 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:09.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:09.664 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:09.754 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:15:09.754 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 274 (0x7f431bfc7910) [ CSSD]2013-05-23 17:15:09.754 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7910 (RPC#274) tag(112002d) sent to node 1 [ CSSD]2013-05-23 17:15:09.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:09.755 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 274 (0x7f431bfc7910) [ CSSD]2013-05-23 17:15:09.755 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7910 (RPC#274) state 6, flags 0x202 [ CSSD]2013-05-23 17:15:09.755 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7910, ret 0, client (nil) [ CSSD]2013-05-23 17:15:09.755 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 274 [ CSSD]2013-05-23 17:15:10.621 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:10.622 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:15:11.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:11.688 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:11.688 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:13.670 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:13.693 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:13.693 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:14.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:14.482 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015110, gin 1, member 0 [ CSSD]2013-05-23 17:15:14.482 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:15:14.482 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:15:14.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:14.489 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:15:14.489 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c13ef60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:15:15.625 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:15.625 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:15.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:15.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:15.726 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:17.684 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:17.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:17.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:19.695 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:19.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:19.743 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:19.746 [1191196992] >TRACE: clssgmAllocProc: (0x2303710) allocated [ CSSD]2013-05-23 17:15:19.747 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:15:19.748 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2303710 - 1,2,3 [ CSSD]2013-05-23 17:15:19.748 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x2303710) pid(17903) version 11:1:1:4 [ CSSD]2013-05-23 17:15:19.754 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2303710), client(1/0x22ff870) [ CSSD]2013-05-23 17:15:19.754 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:15:19.754 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ff870 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 17:15:19.754 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:19.754 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 275 (0x7f431bfc7990) [ CSSD]2013-05-23 17:15:19.754 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7990 (RPC#275) tag(113002d) sent to node 1 [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0113002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015111, gin 1, member 0 [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7990 (RPC#275) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7990, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff870 [ CSSD]2013-05-23 17:15:19.755 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 275 [ CSSD]2013-05-23 17:15:19.761 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ff870) [ CSSD]2013-05-23 17:15:19.761 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ff870), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:19.761 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc5a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:19.761 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 276 (0x7f431bfc7a10) [ CSSD]2013-05-23 17:15:19.761 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7a10 (RPC#276) tag(114002d) sent to node 1 [ CSSD]2013-05-23 17:15:19.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0114002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:19.762 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:19.762 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7a10 (RPC#276) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:19.762 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7a10, ret 0, client 0x22ff870 member 0x22fc5a0 [ CSSD]2013-05-23 17:15:19.762 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 276 [ CSSD]2013-05-23 17:15:19.762 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc5a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:15:19.780 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:15:19.781 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 277 (0x7f431bfc7a90) [ CSSD]2013-05-23 17:15:19.781 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7a90 (RPC#277) tag(115002d) sent to node 1 [ CSSD]2013-05-23 17:15:19.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:19.781 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 277 (0x7f431bfc7a90) [ CSSD]2013-05-23 17:15:19.781 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7a90 (RPC#277) state 6, flags 0x202 [ CSSD]2013-05-23 17:15:19.781 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7a90, ret 0, client (nil) [ CSSD]2013-05-23 17:15:19.782 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 277 [ CSSD]2013-05-23 17:15:19.964 [1191196992] >TRACE: clssgmDeadProc: proc 0x2303710 [ CSSD]2013-05-23 17:15:19.964 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2303710) con(0x230f280) skgpid ospid 17903 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:20.022 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c109f80) [ CSSD]2013-05-23 17:15:20.022 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c109f80), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:15:20.022 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c109f80), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:15:20.023 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 17:15:20.033 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:15:20.033 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c108e70), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:15:20.033 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c108e70), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:15:20.033 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:15:20.033 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e5840 [ CSSD]2013-05-23 17:15:20.033 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e5840) con(0x7f431c12a550) skgpid 12568 ospid 12568 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:20.628 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:20.629 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:21.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:21.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:21.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:23.163 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:15:23.164 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:15:23.165 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:15:23.165 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f280) proc(0x22d6cf0) pid(17956/17956) version 11:1:1:4 [ CSSD]2013-05-23 17:15:23.171 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d6cf0), client(1/0x22f8860) [ CSSD]2013-05-23 17:15:23.171 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f8860) [ CSSD]2013-05-23 17:15:23.171 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:15:23.171 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:15:23.751 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:23.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:23.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:25.632 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:25.632 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:25.758 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:25.765 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:25.765 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:27.765 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:27.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:27.771 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:29.635 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:29.635 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:15:29.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:29.652 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015112, gin 1, member 0 [ CSSD]2013-05-23 17:15:29.652 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:15:29.652 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:15:29.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:29.660 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:15:29.660 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:15:29.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:29.735 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015113, gin 1, member 0 [ CSSD]2013-05-23 17:15:29.735 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:15:29.735 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:15:29.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:29.742 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:15:29.742 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:15:29.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:29.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:29.775 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:29.839 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:15:29.840 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 278 (0x7f431bfc7b10) [ CSSD]2013-05-23 17:15:29.840 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7b10 (RPC#278) tag(116002d) sent to node 1 [ CSSD]2013-05-23 17:15:29.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:29.840 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 278 (0x7f431bfc7b10) [ CSSD]2013-05-23 17:15:29.840 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7b10 (RPC#278) state 6, flags 0x202 [ CSSD]2013-05-23 17:15:29.840 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7b10, ret 0, client (nil) [ CSSD]2013-05-23 17:15:29.841 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 278 [ CSSD]2013-05-23 17:15:31.782 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:31.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:31.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:33.517 [1191196992] >TRACE: clssgmAllocProc: (0x205e170) allocated [ CSSD]2013-05-23 17:15:33.519 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:15:33.519 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e170 - 1,2,3 [ CSSD]2013-05-23 17:15:33.519 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e170) pid(18044) version 11:1:1:4 [ CSSD]2013-05-23 17:15:33.536 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205e170), client(1/0x22d7610) [ CSSD]2013-05-23 17:15:33.536 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7610) [ CSSD]2013-05-23 17:15:33.536 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7610 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:15:33.536 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:33.536 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 279 (0x7f431bfc7b90) [ CSSD]2013-05-23 17:15:33.536 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7b90 (RPC#279) tag(117002d) sent to node 1 [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0117002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015114, gin 1, member 0 [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee3f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7b90 (RPC#279) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7b90, ret 0, client 0x22d7610 member 0x22ee6f0 [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7610 [ CSSD]2013-05-23 17:15:33.537 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 279 [ CSSD]2013-05-23 17:15:33.544 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7610) [ CSSD]2013-05-23 17:15:33.544 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7610), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:33.544 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ee6f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:33.544 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 280 (0x7f431bfc7c10) [ CSSD]2013-05-23 17:15:33.544 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7c10 (RPC#280) tag(118002d) sent to node 1 [ CSSD]2013-05-23 17:15:33.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0118002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:33.545 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:33.545 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7c10 (RPC#280) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:33.545 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7c10, ret 0, client 0x22d7610 member 0x22ee6f0 [ CSSD]2013-05-23 17:15:33.545 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 280 [ CSSD]2013-05-23 17:15:33.545 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ee6f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:15:33.747 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e170 [ CSSD]2013-05-23 17:15:33.747 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e170) con(0x22ff670) skgpid ospid 18044 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:33.792 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:33.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:33.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:34.639 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:34.639 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:35.000 [1191196992] >TRACE: clssgmAllocProc: (0x205e170) allocated [ CSSD]2013-05-23 17:15:35.001 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:15:35.002 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e170 - 1,2,3 [ CSSD]2013-05-23 17:15:35.002 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e170) pid(18091) version 11:1:1:4 [ CSSD]2013-05-23 17:15:35.018 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205e170), client(1/0x22d7610) [ CSSD]2013-05-23 17:15:35.019 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7610) [ CSSD]2013-05-23 17:15:35.019 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7610 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 17:15:35.019 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:35.019 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 281 (0x7f431bfc7c90) [ CSSD]2013-05-23 17:15:35.019 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7c90 (RPC#281) tag(119002d) sent to node 1 [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0119002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015115, gin 1, member 0 [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee3f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7c90 (RPC#281) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7c90, ret 0, client 0x22d7610 member 0x22ee6f0 [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7610 [ CSSD]2013-05-23 17:15:35.019 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 281 [ CSSD]2013-05-23 17:15:35.026 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7610) [ CSSD]2013-05-23 17:15:35.026 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7610), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:35.026 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ee6f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:35.026 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 282 (0x7f431bfc7d10) [ CSSD]2013-05-23 17:15:35.026 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7d10 (RPC#282) tag(11a002d) sent to node 1 [ CSSD]2013-05-23 17:15:35.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(011a002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:35.026 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:35.026 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7d10 (RPC#282) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:35.026 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7d10, ret 0, client 0x22d7610 member 0x22ee6f0 [ CSSD]2013-05-23 17:15:35.026 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 282 [ CSSD]2013-05-23 17:15:35.026 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ee6f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:15:35.229 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e170 [ CSSD]2013-05-23 17:15:35.229 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e170) con(0x22ff670) skgpid ospid 18091 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:35.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:35.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:35.826 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:37.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:37.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:37.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:39.643 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:39.643 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:39.845 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:39.845 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:39.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:39.872 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:15:39.872 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 283 (0x7f431bfc7d90) [ CSSD]2013-05-23 17:15:39.872 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7d90 (RPC#283) tag(11b002d) sent to node 1 [ CSSD]2013-05-23 17:15:39.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:39.873 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 283 (0x7f431bfc7d90) [ CSSD]2013-05-23 17:15:39.873 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7d90 (RPC#283) state 6, flags 0x202 [ CSSD]2013-05-23 17:15:39.873 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7d90, ret 0, client (nil) [ CSSD]2013-05-23 17:15:39.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 283 [ CSSD]2013-05-23 17:15:41.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:41.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:41.890 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:43.646 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:43.646 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:15:43.873 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:43.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:43.896 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:44.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:44.985 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015116, gin 1, member 0 [ CSSD]2013-05-23 17:15:44.985 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:15:44.985 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:15:44.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:44.991 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:15:44.991 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:15:45.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:45.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:45.948 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:47.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:47.973 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:47.973 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:48.650 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:48.650 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:49.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:49.909 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:15:49.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 284 (0x7f431bfc7e10) [ CSSD]2013-05-23 17:15:49.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7e10 (RPC#284) tag(11c002d) sent to node 1 [ CSSD]2013-05-23 17:15:49.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:49.910 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 284 (0x7f431bfc7e10) [ CSSD]2013-05-23 17:15:49.910 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7e10 (RPC#284) state 6, flags 0x202 [ CSSD]2013-05-23 17:15:49.910 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7e10, ret 0, client (nil) [ CSSD]2013-05-23 17:15:49.910 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 284 [ CSSD]2013-05-23 17:15:49.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:49.986 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:50.264 [1191196992] >TRACE: clssgmAllocProc: (0x22ff820) allocated [ CSSD]2013-05-23 17:15:50.265 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:15:50.266 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ff820 - 1,2,3 [ CSSD]2013-05-23 17:15:50.266 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dfd50) proc(0x22ff820) pid(18234) version 11:1:1:4 [ CSSD]2013-05-23 17:15:50.272 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22ff820), client(1/0x22d7550) [ CSSD]2013-05-23 17:15:50.272 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7550) [ CSSD]2013-05-23 17:15:50.272 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7550 with con 0x205e170, requested num -1 [ CSSD]2013-05-23 17:15:50.272 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:50.272 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 285 (0x7f431bfc7e90) [ CSSD]2013-05-23 17:15:50.272 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7e90 (RPC#285) tag(11d002d) sent to node 1 [ CSSD]2013-05-23 17:15:50.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(011d002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:50.272 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015117, gin 1, member 0 [ CSSD]2013-05-23 17:15:50.272 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:15:50.273 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:15:50.273 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:15:50.273 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7e90 (RPC#285) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:50.273 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc7e90, ret 0, client 0x22d7550 member 0x22ee070 [ CSSD]2013-05-23 17:15:50.273 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7550 [ CSSD]2013-05-23 17:15:50.273 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 285 [ CSSD]2013-05-23 17:15:50.280 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7550) [ CSSD]2013-05-23 17:15:50.280 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7550), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:50.280 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ee070) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:15:50.280 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 286 (0x7f431bfc7f10) [ CSSD]2013-05-23 17:15:50.280 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7f10 (RPC#286) tag(11e002d) sent to node 1 [ CSSD]2013-05-23 17:15:50.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(011e002d) incarnation(174631051) [ CSSD]2013-05-23 17:15:50.280 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:15:50.280 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7f10 (RPC#286) state 6, flags 0x100 [ CSSD]2013-05-23 17:15:50.280 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc7f10, ret 0, client 0x22d7550 member 0x22ee070 [ CSSD]2013-05-23 17:15:50.281 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 286 [ CSSD]2013-05-23 17:15:50.281 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ee070) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:15:50.483 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ff820 [ CSSD]2013-05-23 17:15:50.483 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ff820) con(0x22dfd50) skgpid ospid 18234 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:15:51.900 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:51.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:51.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:53.653 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:53.653 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:53.953 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:53.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:53.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:55.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:55.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:55.988 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:57.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:15:57.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:57.994 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:15:58.656 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:15:58.657 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:15:59.966 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:15:59.966 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 287 (0x7f431bfc7f90) [ CSSD]2013-05-23 17:15:59.966 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc7f90 (RPC#287) tag(11f002d) sent to node 1 [ CSSD]2013-05-23 17:15:59.967 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:15:59.967 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 287 (0x7f431bfc7f90) [ CSSD]2013-05-23 17:15:59.967 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc7f90 (RPC#287) state 6, flags 0x202 [ CSSD]2013-05-23 17:15:59.967 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc7f90, ret 0, client (nil) [ CSSD]2013-05-23 17:15:59.967 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 287 [ CSSD]2013-05-23 17:15:59.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:00.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:00.046 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:00.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:00.245 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015118, gin 1, member 0 [ CSSD]2013-05-23 17:16:00.245 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:16:00.245 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:16:00.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:00.252 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:16:00.252 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:16:01.983 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:02.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:02.052 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:02.659 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:02.659 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:16:03.993 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:04.077 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:04.077 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:05.521 [1191196992] >TRACE: clssgmAllocProc: (0x205e170) allocated [ CSSD]2013-05-23 17:16:05.522 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:16:05.522 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e170 - 1,2,3 [ CSSD]2013-05-23 17:16:05.522 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f8750) proc(0x205e170) pid(18385) version 11:1:1:4 [ CSSD]2013-05-23 17:16:05.528 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205e170), client(1/0x22ee320) [ CSSD]2013-05-23 17:16:05.528 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee320) [ CSSD]2013-05-23 17:16:05.528 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22ee320 with con 0x22edf80, requested num -1 [ CSSD]2013-05-23 17:16:05.528 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:05.528 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 288 (0x7f431bfc8010) [ CSSD]2013-05-23 17:16:05.528 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8010 (RPC#288) tag(120002d) sent to node 1 [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0120002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015119, gin 1, member 0 [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee490/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8010 (RPC#288) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8010, ret 0, client 0x22ee320 member 0x205e3d0 [ CSSD]2013-05-23 17:16:05.529 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee320 [ CSSD]2013-05-23 17:16:05.530 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 288 [ CSSD]2013-05-23 17:16:05.536 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee320) [ CSSD]2013-05-23 17:16:05.536 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee320), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:05.536 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e3d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:05.536 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 289 (0x7f431bfc8090) [ CSSD]2013-05-23 17:16:05.536 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8090 (RPC#289) tag(121002d) sent to node 1 [ CSSD]2013-05-23 17:16:05.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0121002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:05.536 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:05.536 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8090 (RPC#289) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:05.536 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8090, ret 0, client 0x22ee320 member 0x205e3d0 [ CSSD]2013-05-23 17:16:05.536 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 289 [ CSSD]2013-05-23 17:16:05.536 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e3d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:16:05.739 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e170 [ CSSD]2013-05-23 17:16:05.739 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e170) con(0x22f8750) skgpid ospid 18385 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:06.000 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:06.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:06.089 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:07.663 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:07.663 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:08.051 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:08.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:08.095 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:09.991 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:16:09.992 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 290 (0x7f431bfc8110) [ CSSD]2013-05-23 17:16:09.992 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8110 (RPC#290) tag(122002d) sent to node 1 [ CSSD]2013-05-23 17:16:09.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:09.992 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 290 (0x7f431bfc8110) [ CSSD]2013-05-23 17:16:09.992 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8110 (RPC#290) state 6, flags 0x202 [ CSSD]2013-05-23 17:16:09.992 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8110, ret 0, client (nil) [ CSSD]2013-05-23 17:16:09.992 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 290 [ CSSD]2013-05-23 17:16:10.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:10.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:10.147 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:12.064 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:12.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:12.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:12.666 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:12.667 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:14.074 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:14.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:14.159 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:15.494 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:15.494 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015120, gin 1, member 0 [ CSSD]2013-05-23 17:16:15.495 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:16:15.495 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:16:15.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:15.502 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:16:15.502 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:16:16.081 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:16.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:16.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:17.670 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:17.671 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:18.091 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:18.196 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:18.196 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:20.017 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:16:20.018 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 291 (0x7f431bfc8190) [ CSSD]2013-05-23 17:16:20.018 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8190 (RPC#291) tag(123002d) sent to node 1 [ CSSD]2013-05-23 17:16:20.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:20.018 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 291 (0x7f431bfc8190) [ CSSD]2013-05-23 17:16:20.018 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8190 (RPC#291) state 6, flags 0x202 [ CSSD]2013-05-23 17:16:20.018 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8190, ret 0, client (nil) [ CSSD]2013-05-23 17:16:20.018 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 291 [ CSSD]2013-05-23 17:16:20.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:20.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:20.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:20.602 [1191196992] >TRACE: clssgmAllocProc: (0x205e170) allocated [ CSSD]2013-05-23 17:16:20.604 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:16:20.604 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e170 - 1,2,3 [ CSSD]2013-05-23 17:16:20.604 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f8750) proc(0x205e170) pid(18549) version 11:1:1:4 [ CSSD]2013-05-23 17:16:20.611 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x205e170), client(1/0x22ee320) [ CSSD]2013-05-23 17:16:20.611 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22ee320) [ CSSD]2013-05-23 17:16:20.611 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22ee320 with con 0x22edf80, requested num -1 [ CSSD]2013-05-23 17:16:20.611 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:16:20.611 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 292 (0x7f431bfc8210) [ CSSD]2013-05-23 17:16:20.611 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8210 (RPC#292) tag(124002d) sent to node 1 [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0124002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015121, gin 1, member 0 [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee490/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8210 (RPC#292) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8210, ret 0, client 0x22ee320 member 0x205e3d0 [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ee320 [ CSSD]2013-05-23 17:16:20.612 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 292 [ CSSD]2013-05-23 17:16:20.619 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22ee320) [ CSSD]2013-05-23 17:16:20.619 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22ee320), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:16:20.619 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e3d0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:16:20.619 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 293 (0x7f431bfc8290) [ CSSD]2013-05-23 17:16:20.619 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8290 (RPC#293) tag(125002d) sent to node 1 [ CSSD]2013-05-23 17:16:20.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0125002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:20.619 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:16:20.619 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8290 (RPC#293) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:20.619 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8290, ret 0, client 0x22ee320 member 0x205e3d0 [ CSSD]2013-05-23 17:16:20.619 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 293 [ CSSD]2013-05-23 17:16:20.619 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x205e3d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:16:20.730 [1191196992] >TRACE: clssgmAllocProc: (0x22e4ad0) allocated [ CSSD]2013-05-23 17:16:20.731 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:16:20.732 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4ad0 - 1,2,3 [ CSSD]2013-05-23 17:16:20.732 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7350) proc(0x22e4ad0) pid(18554) version 11:1:1:4 [ CSSD]2013-05-23 17:16:20.747 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:16:20.749 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:16:20.752 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4ad0 [ CSSD]2013-05-23 17:16:20.752 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4ad0) con(0x22d7350) skgpid ospid 18554 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:20.776 [1191196992] >TRACE: clssgmAllocProc: (0x22e5380) allocated [ CSSD]2013-05-23 17:16:20.777 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:16:20.778 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5380 - 1,2,3 [ CSSD]2013-05-23 17:16:20.778 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7350) proc(0x22e5380) pid(18560) version 11:1:1:4 [ CSSD]2013-05-23 17:16:20.784 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e5380), client(1/0x22d7650) [ CSSD]2013-05-23 17:16:20.784 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7650) [ CSSD]2013-05-23 17:16:20.784 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7650 with con 0x22e46f0, requested num -1 [ CSSD]2013-05-23 17:16:20.784 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:20.784 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 294 (0x7f431bfc8310) [ CSSD]2013-05-23 17:16:20.784 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8310 (RPC#294) tag(126002d) sent to node 1 [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0126002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015122, gin 1, member 0 [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2320f80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8310 (RPC#294) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8310, ret 0, client 0x22d7650 member 0x22e4b30 [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7650 [ CSSD]2013-05-23 17:16:20.785 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 294 [ CSSD]2013-05-23 17:16:20.792 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7650) [ CSSD]2013-05-23 17:16:20.792 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7650), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:20.792 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4b30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:20.792 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 295 (0x7f431bfc8390) [ CSSD]2013-05-23 17:16:20.792 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8390 (RPC#295) tag(127002d) sent to node 1 [ CSSD]2013-05-23 17:16:20.792 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0127002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:20.792 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:20.793 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8390 (RPC#295) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:20.793 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8390, ret 0, client 0x22d7650 member 0x22e4b30 [ CSSD]2013-05-23 17:16:20.793 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 295 [ CSSD]2013-05-23 17:16:20.793 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4b30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:16:20.822 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e170 [ CSSD]2013-05-23 17:16:20.822 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e170) con(0x22f8750) skgpid ospid 18549 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:20.995 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5380 [ CSSD]2013-05-23 17:16:20.995 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5380) con(0x22d7350) skgpid ospid 18560 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:21.673 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:21.674 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:16:22.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:22.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:22.253 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:24.103 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:24.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:24.279 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:24.664 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c135120) [ CSSD]2013-05-23 17:16:24.664 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c135120), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:16:24.664 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c135120), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:16:24.664 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 17:16:24.670 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:16:24.670 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:16:24.670 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:16:24.670 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:16:24.670 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f4f00 [ CSSD]2013-05-23 17:16:24.670 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f4f00) con(0x22f2e50) skgpid 13256 ospid 13256 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:26.110 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:26.370 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:26.370 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:26.677 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:26.677 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:28.162 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:28.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:28.396 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:29.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:29.915 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015123, gin 1, member 0 [ CSSD]2013-05-23 17:16:29.915 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:16:29.915 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:16:29.922 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:29.922 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:16:29.922 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:16:30.042 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:16:30.043 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 296 (0x7f431bfc8410) [ CSSD]2013-05-23 17:16:30.043 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8410 (RPC#296) tag(128002d) sent to node 1 [ CSSD]2013-05-23 17:16:30.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:30.043 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 296 (0x7f431bfc8410) [ CSSD]2013-05-23 17:16:30.043 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8410 (RPC#296) state 6, flags 0x202 [ CSSD]2013-05-23 17:16:30.043 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8410, ret 0, client (nil) [ CSSD]2013-05-23 17:16:30.044 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 296 [ CSSD]2013-05-23 17:16:30.167 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:30.448 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:30.448 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:30.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:30.755 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015124, gin 1, member 0 [ CSSD]2013-05-23 17:16:30.755 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:16:30.755 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:16:30.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:30.762 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:16:30.762 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c124860) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:16:31.681 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:31.681 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:32.174 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:32.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:32.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:33.790 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:16:33.790 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:16:33.791 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:16:33.791 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(18718) version 11:1:1:4 [ CSSD]2013-05-23 17:16:33.798 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x205e360), client(1/0x22f1d30) [ CSSD]2013-05-23 17:16:33.798 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:16:33.798 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22f2e50, requested num -1 [ CSSD]2013-05-23 17:16:33.798 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:33.798 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 297 (0x7f431bfc8490) [ CSSD]2013-05-23 17:16:33.798 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8490 (RPC#297) tag(129002d) sent to node 1 [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0129002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015125, gin 1, member 0 [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8490 (RPC#297) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8490, ret 0, client 0x22f1d30 member 0x205e170 [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:16:33.799 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 297 [ CSSD]2013-05-23 17:16:33.805 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:16:33.806 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:33.806 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e170) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:33.806 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 298 (0x7f431bfc8510) [ CSSD]2013-05-23 17:16:33.806 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8510 (RPC#298) tag(12a002d) sent to node 1 [ CSSD]2013-05-23 17:16:33.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(012a002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:33.806 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:33.806 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8510 (RPC#298) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:33.806 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8510, ret 0, client 0x22f1d30 member 0x205e170 [ CSSD]2013-05-23 17:16:33.806 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 298 [ CSSD]2013-05-23 17:16:33.806 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e170) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:16:34.009 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:16:34.009 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 18718 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:34.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:34.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:34.465 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:36.035 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:16:36.037 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:16:36.037 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:16:36.037 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(18740) version 11:1:1:4 [ CSSD]2013-05-23 17:16:36.054 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x205e360), client(1/0x22f1d30) [ CSSD]2013-05-23 17:16:36.054 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:16:36.054 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22f2e50, requested num -1 [ CSSD]2013-05-23 17:16:36.054 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:36.054 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 299 (0x7f431bfc8590) [ CSSD]2013-05-23 17:16:36.054 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8590 (RPC#299) tag(12b002d) sent to node 1 [ CSSD]2013-05-23 17:16:36.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(012b002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:36.054 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015126, gin 1, member 0 [ CSSD]2013-05-23 17:16:36.054 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:16:36.054 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:16:36.054 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:16:36.054 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8590 (RPC#299) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:36.055 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8590, ret 0, client 0x22f1d30 member 0x205e170 [ CSSD]2013-05-23 17:16:36.055 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:16:36.055 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 299 [ CSSD]2013-05-23 17:16:36.061 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:16:36.061 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:36.061 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e170) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:36.061 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 300 (0x7f431bfc8610) [ CSSD]2013-05-23 17:16:36.061 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8610 (RPC#300) tag(12c002d) sent to node 1 [ CSSD]2013-05-23 17:16:36.061 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(012c002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:36.061 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:36.061 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8610 (RPC#300) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:36.061 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8610, ret 0, client 0x22f1d30 member 0x205e170 [ CSSD]2013-05-23 17:16:36.062 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 300 [ CSSD]2013-05-23 17:16:36.062 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e170) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:16:36.191 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:36.264 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:16:36.264 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 18740 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:36.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:36.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:36.684 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:36.684 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:38.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:38.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:38.477 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:40.070 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:16:40.070 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 301 (0x7f431bfc8690) [ CSSD]2013-05-23 17:16:40.070 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8690 (RPC#301) tag(12d002d) sent to node 1 [ CSSD]2013-05-23 17:16:40.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:40.070 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 301 (0x7f431bfc8690) [ CSSD]2013-05-23 17:16:40.070 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8690 (RPC#301) state 6, flags 0x202 [ CSSD]2013-05-23 17:16:40.070 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8690, ret 0, client (nil) [ CSSD]2013-05-23 17:16:40.071 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 301 [ CSSD]2013-05-23 17:16:40.206 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:40.548 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:40.548 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:41.687 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:41.688 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:42.212 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:42.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:42.574 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:44.265 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:44.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:44.584 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:46.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:46.005 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015127, gin 1, member 0 [ CSSD]2013-05-23 17:16:46.005 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:16:46.005 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:16:46.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:46.012 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:16:46.012 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:16:46.272 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:46.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:46.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:46.691 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:46.692 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:48.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:48.596 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:48.596 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:50.093 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:16:50.093 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 302 (0x7f431bfc8710) [ CSSD]2013-05-23 17:16:50.093 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8710 (RPC#302) tag(12e002d) sent to node 1 [ CSSD]2013-05-23 17:16:50.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:50.094 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 302 (0x7f431bfc8710) [ CSSD]2013-05-23 17:16:50.094 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8710 (RPC#302) state 6, flags 0x202 [ CSSD]2013-05-23 17:16:50.094 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8710, ret 0, client (nil) [ CSSD]2013-05-23 17:16:50.094 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 302 [ CSSD]2013-05-23 17:16:50.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:50.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:50.618 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:51.298 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:16:51.341 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:16:51.341 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:16:51.341 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(18885) version 11:1:1:4 [ CSSD]2013-05-23 17:16:51.348 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x205e360), client(1/0x22f2c60) [ CSSD]2013-05-23 17:16:51.348 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:16:51.348 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2c60 with con 0x22f2f40, requested num -1 [ CSSD]2013-05-23 17:16:51.348 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:51.348 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 303 (0x7f431bfc8790) [ CSSD]2013-05-23 17:16:51.348 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8790 (RPC#303) tag(12f002d) sent to node 1 [ CSSD]2013-05-23 17:16:51.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(012f002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:51.348 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015128, gin 1, member 0 [ CSSD]2013-05-23 17:16:51.348 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:16:51.349 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:16:51.349 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:16:51.349 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8790 (RPC#303) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:51.349 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8790, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:16:51.349 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2c60 [ CSSD]2013-05-23 17:16:51.349 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 303 [ CSSD]2013-05-23 17:16:51.356 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:16:51.356 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2c60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:51.356 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e240) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:16:51.356 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 304 (0x7f431bfc8810) [ CSSD]2013-05-23 17:16:51.356 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8810 (RPC#304) tag(130002d) sent to node 1 [ CSSD]2013-05-23 17:16:51.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0130002d) incarnation(174631051) [ CSSD]2013-05-23 17:16:51.357 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:16:51.357 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8810 (RPC#304) state 6, flags 0x100 [ CSSD]2013-05-23 17:16:51.357 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8810, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:16:51.357 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 304 [ CSSD]2013-05-23 17:16:51.357 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e240) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:16:51.559 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:16:51.559 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 18885 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:16:51.695 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:51.696 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:16:52.295 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:52.624 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:52.624 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:54.306 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:54.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:54.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:55.698 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:16:55.699 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:16:56.312 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:56.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:56.653 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:16:58.363 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:16:58.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:16:58.659 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:00.118 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:17:00.119 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 305 (0x7f431bfc8890) [ CSSD]2013-05-23 17:17:00.119 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8890 (RPC#305) tag(131002d) sent to node 1 [ CSSD]2013-05-23 17:17:00.119 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:00.119 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 305 (0x7f431bfc8890) [ CSSD]2013-05-23 17:17:00.119 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8890 (RPC#305) state 6, flags 0x202 [ CSSD]2013-05-23 17:17:00.119 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8890, ret 0, client (nil) [ CSSD]2013-05-23 17:17:00.120 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 305 [ CSSD]2013-05-23 17:17:00.370 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:00.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:00.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:00.702 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:00.702 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:01.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:01.256 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015129, gin 1, member 0 [ CSSD]2013-05-23 17:17:01.256 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:01.256 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:01.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:01.262 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:01.262 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c124860) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:02.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:02.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:02.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:04.387 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:04.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:04.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:05.706 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:05.706 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:06.394 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:06.595 [1191196992] >TRACE: clssgmAllocProc: (0x22f3040) allocated [ CSSD]2013-05-23 17:17:06.596 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:17:06.596 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3040 - 1,2,3 [ CSSD]2013-05-23 17:17:06.596 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff750) proc(0x22f3040) pid(19042) version 11:1:1:4 [ CSSD]2013-05-23 17:17:06.604 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f3040), client(1/0x22f2c60) [ CSSD]2013-05-23 17:17:06.604 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:06.604 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2c60 with con 0x22f2e50, requested num -1 [ CSSD]2013-05-23 17:17:06.604 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:06.604 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 306 (0x7f431bfc8910) [ CSSD]2013-05-23 17:17:06.604 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8910 (RPC#306) tag(132002d) sent to node 1 [ CSSD]2013-05-23 17:17:06.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0132002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:06.604 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015130, gin 1, member 0 [ CSSD]2013-05-23 17:17:06.604 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee3f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:17:06.605 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:17:06.605 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:17:06.605 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8910 (RPC#306) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:06.605 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8910, ret 0, client 0x22f2c60 member 0x22ee6f0 [ CSSD]2013-05-23 17:17:06.605 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2c60 [ CSSD]2013-05-23 17:17:06.605 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 306 [ CSSD]2013-05-23 17:17:06.613 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:06.613 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2c60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:06.613 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ee6f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:06.613 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 307 (0x7f431bfc8990) [ CSSD]2013-05-23 17:17:06.613 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8990 (RPC#307) tag(133002d) sent to node 1 [ CSSD]2013-05-23 17:17:06.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0133002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:06.614 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:06.614 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8990 (RPC#307) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:06.614 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8990, ret 0, client 0x22f2c60 member 0x22ee6f0 [ CSSD]2013-05-23 17:17:06.614 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 307 [ CSSD]2013-05-23 17:17:06.614 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ee6f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:17:06.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:06.712 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:06.816 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3040 [ CSSD]2013-05-23 17:17:06.816 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3040) con(0x22ff750) skgpid ospid 19042 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:08.404 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:08.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:08.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:09.709 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:09.709 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:17:10.144 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:17:10.145 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 308 (0x7f431bfc8a10) [ CSSD]2013-05-23 17:17:10.145 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8a10 (RPC#308) tag(134002d) sent to node 1 [ CSSD]2013-05-23 17:17:10.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:10.145 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 308 (0x7f431bfc8a10) [ CSSD]2013-05-23 17:17:10.145 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8a10 (RPC#308) state 6, flags 0x202 [ CSSD]2013-05-23 17:17:10.145 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8a10, ret 0, client (nil) [ CSSD]2013-05-23 17:17:10.146 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 308 [ CSSD]2013-05-23 17:17:10.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:10.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:10.770 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:12.417 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:12.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:12.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:13.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:13.949 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015131, gin 1, member 0 [ CSSD]2013-05-23 17:17:13.949 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:13.949 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:13.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:13.957 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:13.957 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:14.466 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:14.713 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:14.713 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:14.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:14.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:16.472 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:16.510 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:16.510 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015132, gin 1, member 0 [ CSSD]2013-05-23 17:17:16.510 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:16.510 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:16.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:16.517 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:16.517 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c124860) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:16.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:16.793 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:18.564 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:18.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:18.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:19.250 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:17:19.251 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:17:19.252 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:17:19.252 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(19376) version 11:1:1:4 [ CSSD]2013-05-23 17:17:19.383 [1191196992] >TRACE: clssgmAllocProc: (0x22eedd0) allocated [ CSSD]2013-05-23 17:17:19.384 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:17:19.384 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22eedd0 - 1,2,3 [ CSSD]2013-05-23 17:17:19.384 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f30f0) proc(0x22eedd0) pid(19386) version 11:1:1:4 [ CSSD]2013-05-23 17:17:19.399 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:17:19.401 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:17:19.403 [1191196992] >TRACE: clssgmDeadProc: proc 0x22eedd0 [ CSSD]2013-05-23 17:17:19.403 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22eedd0) con(0x22f30f0) skgpid ospid 19386 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:19.482 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(78/0x22ff8b0) [ CSSD]2013-05-23 17:17:19.482 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 78 (0x22ff8b0) [ CSSD]2013-05-23 17:17:19.482 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22ff8b0 with con 0x22f30f0, requested num -1 [ CSSD]2013-05-23 17:17:19.482 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:17:19.482 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 309 (0x7f431bfc8a90) [ CSSD]2013-05-23 17:17:19.482 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8a90 (RPC#309) tag(135002d) sent to node 1 [ CSSD]2013-05-23 17:17:19.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0135002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:19.482 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015133, gin 1, member 0 [ CSSD]2013-05-23 17:17:19.482 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x22ec570/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:17:19.483 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:17:19.483 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:17:19.483 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8a90 (RPC#309) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:19.483 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8a90, ret 0, client 0x22ff8b0 member 0x205e230 [ CSSD]2013-05-23 17:17:19.483 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22ff8b0 [ CSSD]2013-05-23 17:17:19.483 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 309 [ CSSD]2013-05-23 17:17:19.490 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 78 (0x22ff8b0) [ CSSD]2013-05-23 17:17:19.490 [1191196992] >TRACE: clssgmExitGrock: client 78 (0x22ff8b0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:17:19.490 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e230) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:17:19.490 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 310 (0x7f431bfc8b10) [ CSSD]2013-05-23 17:17:19.490 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8b10 (RPC#310) tag(136002d) sent to node 1 [ CSSD]2013-05-23 17:17:19.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0136002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:19.490 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:17:19.490 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8b10 (RPC#310) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:19.490 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8b10, ret 0, client 0x22ff8b0 member 0x205e230 [ CSSD]2013-05-23 17:17:19.491 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 310 [ CSSD]2013-05-23 17:17:19.491 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x205e230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:17:19.492 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:17:19.492 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 19376 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:19.716 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:19.716 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:20.171 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:17:20.172 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 311 (0x7f431bfc8b90) [ CSSD]2013-05-23 17:17:20.172 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8b90 (RPC#311) tag(137002d) sent to node 1 [ CSSD]2013-05-23 17:17:20.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:20.172 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 311 (0x7f431bfc8b90) [ CSSD]2013-05-23 17:17:20.172 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8b90 (RPC#311) state 6, flags 0x202 [ CSSD]2013-05-23 17:17:20.172 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8b90, ret 0, client (nil) [ CSSD]2013-05-23 17:17:20.173 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 311 [ CSSD]2013-05-23 17:17:20.571 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:20.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:20.890 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:21.864 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:17:21.865 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:17:21.865 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:17:21.865 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(19416) version 11:1:1:4 [ CSSD]2013-05-23 17:17:21.882 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x205e360), client(1/0x22f2c60) [ CSSD]2013-05-23 17:17:21.882 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:21.882 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2c60 with con 0x22f2f40, requested num -1 [ CSSD]2013-05-23 17:17:21.882 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:21.882 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 312 (0x7f431bfc8c10) [ CSSD]2013-05-23 17:17:21.882 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8c10 (RPC#312) tag(138002d) sent to node 1 [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0138002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015134, gin 1, member 0 [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8c10 (RPC#312) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8c10, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:17:21.882 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2c60 [ CSSD]2013-05-23 17:17:21.883 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 312 [ CSSD]2013-05-23 17:17:21.889 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:21.889 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2c60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:21.889 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e240) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:21.889 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 313 (0x7f431bfc8c90) [ CSSD]2013-05-23 17:17:21.889 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8c90 (RPC#313) tag(139002d) sent to node 1 [ CSSD]2013-05-23 17:17:21.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0139002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:21.889 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:21.889 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8c90 (RPC#313) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:21.889 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8c90, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:17:21.889 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 313 [ CSSD]2013-05-23 17:17:21.889 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e240) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:17:22.092 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:17:22.092 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 19416 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:22.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:22.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:22.915 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:24.587 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:24.720 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:24.720 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:24.967 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:24.967 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:26.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:26.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:26.974 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:28.604 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:28.723 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:28.724 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:17:28.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:28.999 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:30.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:30.170 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015135, gin 1, member 0 [ CSSD]2013-05-23 17:17:30.170 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:30.170 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:30.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:30.177 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:30.177 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:30.196 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:17:30.197 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 314 (0x7f431bfc8d10) [ CSSD]2013-05-23 17:17:30.197 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8d10 (RPC#314) tag(13a002d) sent to node 1 [ CSSD]2013-05-23 17:17:30.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:30.197 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 314 (0x7f431bfc8d10) [ CSSD]2013-05-23 17:17:30.197 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8d10 (RPC#314) state 6, flags 0x202 [ CSSD]2013-05-23 17:17:30.197 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8d10, ret 0, client (nil) [ CSSD]2013-05-23 17:17:30.197 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 314 [ CSSD]2013-05-23 17:17:30.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:31.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:31.029 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:31.761 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:31.761 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015136, gin 1, member 0 [ CSSD]2013-05-23 17:17:31.761 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:31.761 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:31.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:31.767 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:31.767 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c124860) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:32.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:33.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:33.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:33.727 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:33.727 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:34.048 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:17:34.049 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:17:34.049 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:17:34.049 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(19537) version 11:1:1:4 [ CSSD]2013-05-23 17:17:34.056 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x205e360), client(1/0x22f2c60) [ CSSD]2013-05-23 17:17:34.056 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:34.056 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2c60 with con 0x22f2f40, requested num -1 [ CSSD]2013-05-23 17:17:34.056 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:34.056 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 315 (0x7f431bfc8d90) [ CSSD]2013-05-23 17:17:34.056 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8d90 (RPC#315) tag(13b002d) sent to node 1 [ CSSD]2013-05-23 17:17:34.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(013b002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:34.056 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015137, gin 1, member 0 [ CSSD]2013-05-23 17:17:34.056 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:17:34.057 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:17:34.057 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:17:34.057 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8d90 (RPC#315) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:34.057 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8d90, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:17:34.057 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2c60 [ CSSD]2013-05-23 17:17:34.057 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 315 [ CSSD]2013-05-23 17:17:34.064 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:34.064 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2c60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:34.064 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e240) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:34.064 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 316 (0x7f431bfc8e10) [ CSSD]2013-05-23 17:17:34.064 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8e10 (RPC#316) tag(13c002d) sent to node 1 [ CSSD]2013-05-23 17:17:34.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(013c002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:34.064 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:34.064 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8e10 (RPC#316) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:34.064 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8e10, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:17:34.064 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 316 [ CSSD]2013-05-23 17:17:34.064 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e240) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:17:34.267 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:17:34.267 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 19537 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:34.668 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:35.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:35.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:36.675 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:37.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:37.074 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:37.133 [1191196992] >TRACE: clssgmAllocProc: (0x205e360) allocated [ CSSD]2013-05-23 17:17:37.135 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:17:37.135 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e360 - 1,2,3 [ CSSD]2013-05-23 17:17:37.135 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ff670) proc(0x205e360) pid(19570) version 11:1:1:4 [ CSSD]2013-05-23 17:17:37.142 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x205e360), client(1/0x22f2c60) [ CSSD]2013-05-23 17:17:37.142 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:37.142 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2c60 with con 0x22f2f40, requested num -1 [ CSSD]2013-05-23 17:17:37.142 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:37.142 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 317 (0x7f431bfc8e90) [ CSSD]2013-05-23 17:17:37.142 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8e90 (RPC#317) tag(13d002d) sent to node 1 [ CSSD]2013-05-23 17:17:37.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(013d002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:37.142 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015138, gin 1, member 0 [ CSSD]2013-05-23 17:17:37.142 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ee320/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:17:37.143 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:17:37.143 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:17:37.143 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8e90 (RPC#317) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:37.143 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc8e90, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:17:37.143 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2c60 [ CSSD]2013-05-23 17:17:37.143 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 317 [ CSSD]2013-05-23 17:17:37.148 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2c60) [ CSSD]2013-05-23 17:17:37.148 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2c60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:37.148 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x205e240) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:37.148 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 318 (0x7f431bfc8f10) [ CSSD]2013-05-23 17:17:37.148 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8f10 (RPC#318) tag(13e002d) sent to node 1 [ CSSD]2013-05-23 17:17:37.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(013e002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:37.149 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:37.149 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8f10 (RPC#318) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:37.149 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc8f10, ret 0, client 0x22f2c60 member 0x205e240 [ CSSD]2013-05-23 17:17:37.149 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 318 [ CSSD]2013-05-23 17:17:37.149 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x205e240) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:17:37.351 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e360 [ CSSD]2013-05-23 17:17:37.351 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e360) con(0x22ff670) skgpid ospid 19570 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:38.676 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:38.731 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:38.731 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:39.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:39.080 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:40.222 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:17:40.222 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 319 (0x7f431bfc8f90) [ CSSD]2013-05-23 17:17:40.223 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc8f90 (RPC#319) tag(13f002d) sent to node 1 [ CSSD]2013-05-23 17:17:40.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:40.223 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 319 (0x7f431bfc8f90) [ CSSD]2013-05-23 17:17:40.223 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc8f90 (RPC#319) state 6, flags 0x202 [ CSSD]2013-05-23 17:17:40.223 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc8f90, ret 0, client (nil) [ CSSD]2013-05-23 17:17:40.223 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 319 [ CSSD]2013-05-23 17:17:40.682 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:41.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:41.090 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:42.689 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:42.734 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:42.734 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:17:43.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:43.095 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:43.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:43.480 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015139, gin 1, member 0 [ CSSD]2013-05-23 17:17:43.480 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:43.480 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:43.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:43.487 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:43.487 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c124860) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:44.699 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:45.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:45.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:46.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:47.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:47.019 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015140, gin 1, member 0 [ CSSD]2013-05-23 17:17:47.019 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:17:47.019 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:17:47.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:47.025 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:17:47.025 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c129c20) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:17:47.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:47.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:47.738 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:47.738 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:48.716 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:49.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:49.219 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:50.054 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 17:17:50.054 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0cf720), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:17:50.054 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0cf720), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:17:50.055 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 22 [ CSSD]2013-05-23 17:17:50.064 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:17:50.064 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:17:50.064 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:17:50.064 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:17:50.065 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 17:17:50.065 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c0e6e20) skgpid 14439 ospid 14439 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:50.253 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:17:50.254 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 320 (0x7f431bfc9010) [ CSSD]2013-05-23 17:17:50.254 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9010 (RPC#320) tag(140002d) sent to node 1 [ CSSD]2013-05-23 17:17:50.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:50.254 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 320 (0x7f431bfc9010) [ CSSD]2013-05-23 17:17:50.254 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9010 (RPC#320) state 6, flags 0x202 [ CSSD]2013-05-23 17:17:50.254 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9010, ret 0, client (nil) [ CSSD]2013-05-23 17:17:50.255 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 320 [ CSSD]2013-05-23 17:17:50.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:51.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:51.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:52.400 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 17:17:52.401 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:17:52.402 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 17:17:52.402 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c108da0) pid(19780) version 11:1:1:4 [ CSSD]2013-05-23 17:17:52.408 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108da0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:17:52.408 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:17:52.408 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22fc730, requested num -1 [ CSSD]2013-05-23 17:17:52.408 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:52.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 321 (0x7f431bfc9090) [ CSSD]2013-05-23 17:17:52.408 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9090 (RPC#321) tag(141002d) sent to node 1 [ CSSD]2013-05-23 17:17:52.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0141002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015141, gin 1, member 0 [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9090 (RPC#321) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9090, ret 0, client 0x22f1d30 member 0x22dfe20 [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:17:52.409 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 321 [ CSSD]2013-05-23 17:17:52.416 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:17:52.416 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:52.416 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22dfe20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:17:52.416 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 322 (0x7f431bfc9110) [ CSSD]2013-05-23 17:17:52.416 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9110 (RPC#322) tag(142002d) sent to node 1 [ CSSD]2013-05-23 17:17:52.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0142002d) incarnation(174631051) [ CSSD]2013-05-23 17:17:52.417 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:17:52.417 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9110 (RPC#322) state 6, flags 0x100 [ CSSD]2013-05-23 17:17:52.417 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9110, ret 0, client 0x22f1d30 member 0x22dfe20 [ CSSD]2013-05-23 17:17:52.417 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 322 [ CSSD]2013-05-23 17:17:52.417 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22dfe20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:17:52.619 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 17:17:52.619 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c0840c0) skgpid ospid 19780 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:17:52.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:52.742 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:52.742 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:53.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:53.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:54.779 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:55.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:55.288 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:56.786 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:57.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:57.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:17:57.745 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:17:57.746 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:17:58.797 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:17:59.319 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:17:59.319 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:00.281 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:18:00.282 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 323 (0x7f431bfc9190) [ CSSD]2013-05-23 17:18:00.282 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9190 (RPC#323) tag(143002d) sent to node 1 [ CSSD]2013-05-23 17:18:00.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:00.282 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 323 (0x7f431bfc9190) [ CSSD]2013-05-23 17:18:00.282 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9190 (RPC#323) state 6, flags 0x202 [ CSSD]2013-05-23 17:18:00.282 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9190, ret 0, client (nil) [ CSSD]2013-05-23 17:18:00.282 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 323 [ CSSD]2013-05-23 17:18:00.803 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:01.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:01.372 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:01.748 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:01.749 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:18:02.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:02.268 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015142, gin 1, member 0 [ CSSD]2013-05-23 17:18:02.269 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:18:02.269 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:18:02.275 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:02.275 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:18:02.275 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:18:02.809 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:03.280 [1191196992] >TRACE: clssgmAllocProc: (0x205e170) allocated [ CSSD]2013-05-23 17:18:03.281 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:18:03.281 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x205e170 - 1,2,3 [ CSSD]2013-05-23 17:18:03.281 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7350) proc(0x205e170) pid(18638/18638) version 11:1:1:4 [ CSSD]2013-05-23 17:18:03.286 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x205e170), client(1/0x22f1d30) [ CSSD]2013-05-23 17:18:03.286 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:18:03.286 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:18:03.286 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:18:03.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:03.398 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:04.820 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:05.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:05.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:06.752 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:06.753 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:06.826 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:07.415 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:07.415 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:07.655 [1191196992] >TRACE: clssgmAllocProc: (0x22ed180) allocated [ CSSD]2013-05-23 17:18:07.656 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:18:07.657 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed180 - 1,2,3 [ CSSD]2013-05-23 17:18:07.657 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x22ed180) pid(19937) version 11:1:1:4 [ CSSD]2013-05-23 17:18:07.664 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22ed180), client(1/0x2317db0) [ CSSD]2013-05-23 17:18:07.664 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2317db0) [ CSSD]2013-05-23 17:18:07.664 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2317db0 with con 0x22ed2b0, requested num -1 [ CSSD]2013-05-23 17:18:07.664 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:07.664 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 324 (0x7f431bfc9210) [ CSSD]2013-05-23 17:18:07.664 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9210 (RPC#324) tag(144002d) sent to node 1 [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0144002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015143, gin 1, member 0 [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2317f20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9210 (RPC#324) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9210, ret 0, client 0x2317db0 member 0x22ed080 [ CSSD]2013-05-23 17:18:07.664 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2317db0 [ CSSD]2013-05-23 17:18:07.665 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 324 [ CSSD]2013-05-23 17:18:07.671 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2317db0) [ CSSD]2013-05-23 17:18:07.671 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2317db0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:07.671 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ed080) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:07.671 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 325 (0x7f431bfc9290) [ CSSD]2013-05-23 17:18:07.671 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9290 (RPC#325) tag(145002d) sent to node 1 [ CSSD]2013-05-23 17:18:07.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0145002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:07.671 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:07.671 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9290 (RPC#325) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:07.671 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9290, ret 0, client 0x2317db0 member 0x22ed080 [ CSSD]2013-05-23 17:18:07.672 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 325 [ CSSD]2013-05-23 17:18:07.672 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ed080) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:18:07.874 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed180 [ CSSD]2013-05-23 17:18:07.874 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed180) con(0x22dffa0) skgpid ospid 19937 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:18:08.877 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:09.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:09.437 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:10.305 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:18:10.305 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 326 (0x7f431bfc9310) [ CSSD]2013-05-23 17:18:10.306 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9310 (RPC#326) tag(146002d) sent to node 1 [ CSSD]2013-05-23 17:18:10.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:10.306 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 326 (0x7f431bfc9310) [ CSSD]2013-05-23 17:18:10.306 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9310 (RPC#326) state 6, flags 0x202 [ CSSD]2013-05-23 17:18:10.306 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9310, ret 0, client (nil) [ CSSD]2013-05-23 17:18:10.306 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 326 [ CSSD]2013-05-23 17:18:10.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:11.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:11.443 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:11.756 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:11.756 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:12.890 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:13.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:13.449 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:14.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:15.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:15.480 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:15.759 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:15.759 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:18:16.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:17.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:17.486 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:17.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:17.519 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015144, gin 1, member 0 [ CSSD]2013-05-23 17:18:17.519 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:18:17.519 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:18:17.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:17.527 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:18:17.527 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:18:18.917 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:19.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:19.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:20.143 [1191196992] >TRACE: clssgmAllocProc: (0x22ed180) allocated [ CSSD]2013-05-23 17:18:20.144 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:18:20.145 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed180 - 1,2,3 [ CSSD]2013-05-23 17:18:20.145 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x22ed180) pid(19765/19765) version 11:1:1:4 [ CSSD]2013-05-23 17:18:20.161 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22ed180), client(1/0x2317c20) [ CSSD]2013-05-23 17:18:20.161 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2317c20) [ CSSD]2013-05-23 17:18:20.161 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:18:20.161 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:18:20.332 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:18:20.333 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 327 (0x7f431bfc9390) [ CSSD]2013-05-23 17:18:20.333 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9390 (RPC#327) tag(147002d) sent to node 1 [ CSSD]2013-05-23 17:18:20.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:20.333 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 327 (0x7f431bfc9390) [ CSSD]2013-05-23 17:18:20.333 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9390 (RPC#327) state 6, flags 0x202 [ CSSD]2013-05-23 17:18:20.333 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9390, ret 0, client (nil) [ CSSD]2013-05-23 17:18:20.333 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 327 [ CSSD]2013-05-23 17:18:20.763 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:20.763 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:20.924 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:21.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:21.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:22.922 [1191196992] >TRACE: clssgmAllocProc: (0x22e4c70) allocated [ CSSD]2013-05-23 17:18:22.924 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:18:22.924 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4c70 - 1,2,3 [ CSSD]2013-05-23 17:18:22.924 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317d90) proc(0x22e4c70) pid(20099) version 11:1:1:4 [ CSSD]2013-05-23 17:18:22.931 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4c70), client(1/0x22e5390) [ CSSD]2013-05-23 17:18:22.931 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5390) [ CSSD]2013-05-23 17:18:22.931 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5390 with con 0x22e4e90, requested num -1 [ CSSD]2013-05-23 17:18:22.931 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:22.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 328 (0x7f431bfc9410) [ CSSD]2013-05-23 17:18:22.931 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9410 (RPC#328) tag(148002d) sent to node 1 [ CSSD]2013-05-23 17:18:22.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0148002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:22.931 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015145, gin 1, member 0 [ CSSD]2013-05-23 17:18:22.931 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2320f80/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:18:22.931 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:18:22.932 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:18:22.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9410 (RPC#328) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:22.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9410, ret 0, client 0x22e5390 member 0x22e4b60 [ CSSD]2013-05-23 17:18:22.932 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5390 [ CSSD]2013-05-23 17:18:22.932 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 328 [ CSSD]2013-05-23 17:18:22.938 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5390) [ CSSD]2013-05-23 17:18:22.938 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5390), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:22.938 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e4b60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:22.938 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 329 (0x7f431bfc9490) [ CSSD]2013-05-23 17:18:22.938 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9490 (RPC#329) tag(149002d) sent to node 1 [ CSSD]2013-05-23 17:18:22.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0149002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:22.938 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:22.938 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9490 (RPC#329) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:22.938 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9490, ret 0, client 0x22e5390 member 0x22e4b60 [ CSSD]2013-05-23 17:18:22.938 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 329 [ CSSD]2013-05-23 17:18:22.938 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e4b60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:18:22.974 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:23.141 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4c70 [ CSSD]2013-05-23 17:18:23.141 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4c70) con(0x2317d90) skgpid ospid 20099 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:18:23.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:23.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:24.980 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:25.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:25.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:25.766 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:25.766 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:26.986 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:27.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:27.545 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:28.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:29.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:29.578 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:30.358 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:18:30.358 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 330 (0x7f431bfc9510) [ CSSD]2013-05-23 17:18:30.359 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9510 (RPC#330) tag(14a002d) sent to node 1 [ CSSD]2013-05-23 17:18:30.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:30.359 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 330 (0x7f431bfc9510) [ CSSD]2013-05-23 17:18:30.359 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9510 (RPC#330) state 6, flags 0x202 [ CSSD]2013-05-23 17:18:30.359 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9510, ret 0, client (nil) [ CSSD]2013-05-23 17:18:30.359 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 330 [ CSSD]2013-05-23 17:18:30.430 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:30.430 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015146, gin 1, member 0 [ CSSD]2013-05-23 17:18:30.430 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:18:30.430 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:18:30.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:30.437 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:18:30.437 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:18:30.770 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:30.770 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:31.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:31.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:31.605 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:32.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:32.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015147, gin 1, member 0 [ CSSD]2013-05-23 17:18:32.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:18:32.770 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:18:32.777 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:32.777 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:18:32.777 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:18:33.015 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:33.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:33.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:34.307 [1191196992] >TRACE: clssgmAllocProc: (0x22e48f0) allocated [ CSSD]2013-05-23 17:18:34.308 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:18:34.309 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e48f0 - 1,2,3 [ CSSD]2013-05-23 17:18:34.309 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e5370) proc(0x22e48f0) pid(20223) version 11:1:1:4 [ CSSD]2013-05-23 17:18:34.316 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e48f0), client(1/0x22e5070) [ CSSD]2013-05-23 17:18:34.316 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e5070) [ CSSD]2013-05-23 17:18:34.316 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e5070 with con 0x22e4b70, requested num -1 [ CSSD]2013-05-23 17:18:34.316 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:34.316 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 331 (0x7f431bfc9590) [ CSSD]2013-05-23 17:18:34.316 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9590 (RPC#331) tag(14b002d) sent to node 1 [ CSSD]2013-05-23 17:18:34.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(014b002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:34.316 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015148, gin 1, member 0 [ CSSD]2013-05-23 17:18:34.316 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321050/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:18:34.317 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:18:34.317 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:18:34.317 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9590 (RPC#331) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:34.317 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9590, ret 0, client 0x22e5070 member 0x22e47e0 [ CSSD]2013-05-23 17:18:34.317 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e5070 [ CSSD]2013-05-23 17:18:34.317 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 331 [ CSSD]2013-05-23 17:18:34.323 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e5070) [ CSSD]2013-05-23 17:18:34.323 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e5070), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:34.323 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e47e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:34.323 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 332 (0x7f431bfc9610) [ CSSD]2013-05-23 17:18:34.323 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9610 (RPC#332) tag(14c002d) sent to node 1 [ CSSD]2013-05-23 17:18:34.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(014c002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:34.323 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:34.323 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9610 (RPC#332) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:34.323 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9610, ret 0, client 0x22e5070 member 0x22e47e0 [ CSSD]2013-05-23 17:18:34.324 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 332 [ CSSD]2013-05-23 17:18:34.324 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e47e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:18:34.526 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e48f0 [ CSSD]2013-05-23 17:18:34.526 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e48f0) con(0x22e5370) skgpid ospid 20223 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:18:34.773 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:34.773 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:18:35.021 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:35.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:35.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:37.027 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:37.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:37.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:38.178 [1191196992] >TRACE: clssgmAllocProc: (0x22e49c0) allocated [ CSSD]2013-05-23 17:18:38.179 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:18:38.180 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e49c0 - 1,2,3 [ CSSD]2013-05-23 17:18:38.180 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e50a0) proc(0x22e49c0) pid(20282) version 11:1:1:4 [ CSSD]2013-05-23 17:18:38.186 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e49c0), client(1/0x2318ba0) [ CSSD]2013-05-23 17:18:38.186 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2318ba0) [ CSSD]2013-05-23 17:18:38.186 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2318ba0 with con 0x23186a0, requested num -1 [ CSSD]2013-05-23 17:18:38.186 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:38.186 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 333 (0x7f431bfc9690) [ CSSD]2013-05-23 17:18:38.186 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9690 (RPC#333) tag(14d002d) sent to node 1 [ CSSD]2013-05-23 17:18:38.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(014d002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:38.186 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015149, gin 1, member 0 [ CSSD]2013-05-23 17:18:38.186 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321050/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:18:38.187 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:18:38.187 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:18:38.187 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9690 (RPC#333) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:38.187 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9690, ret 0, client 0x2318ba0 member 0x22e48b0 [ CSSD]2013-05-23 17:18:38.187 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2318ba0 [ CSSD]2013-05-23 17:18:38.187 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 333 [ CSSD]2013-05-23 17:18:38.193 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2318ba0) [ CSSD]2013-05-23 17:18:38.193 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2318ba0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:38.193 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e48b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:38.193 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 334 (0x7f431bfc9710) [ CSSD]2013-05-23 17:18:38.193 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9710 (RPC#334) tag(14e002d) sent to node 1 [ CSSD]2013-05-23 17:18:38.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(014e002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:38.193 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:38.193 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9710 (RPC#334) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:38.193 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9710, ret 0, client 0x2318ba0 member 0x22e48b0 [ CSSD]2013-05-23 17:18:38.193 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 334 [ CSSD]2013-05-23 17:18:38.193 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e48b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:18:38.396 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e49c0 [ CSSD]2013-05-23 17:18:38.396 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e49c0) con(0x22e50a0) skgpid ospid 20282 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:18:39.078 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:39.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:39.679 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:39.776 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:39.777 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:40.383 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:18:40.383 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 335 (0x7f431bfc9790) [ CSSD]2013-05-23 17:18:40.383 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9790 (RPC#335) tag(14f002d) sent to node 1 [ CSSD]2013-05-23 17:18:40.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:40.384 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 335 (0x7f431bfc9790) [ CSSD]2013-05-23 17:18:40.384 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9790 (RPC#335) state 6, flags 0x202 [ CSSD]2013-05-23 17:18:40.384 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9790, ret 0, client (nil) [ CSSD]2013-05-23 17:18:40.384 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 335 [ CSSD]2013-05-23 17:18:41.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:41.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:41.723 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:41.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:41.962 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015150, gin 1, member 0 [ CSSD]2013-05-23 17:18:41.962 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:18:41.962 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:18:41.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:41.969 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:18:41.969 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:18:42.098 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:18:42.098 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:18:42.099 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:18:42.099 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:18:42.101 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:18:42.101 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:18:43.094 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:43.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:43.729 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:44.779 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:44.780 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:45.119 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:45.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:45.782 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:47.125 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:47.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:47.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:48.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:48.032 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015151, gin 1, member 0 [ CSSD]2013-05-23 17:18:48.032 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:18:48.033 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:18:48.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:48.039 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:18:48.039 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:18:49.178 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:49.783 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:49.783 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:49.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:49.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:50.408 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:18:50.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 336 (0x7f431bfc9810) [ CSSD]2013-05-23 17:18:50.408 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9810 (RPC#336) tag(150002d) sent to node 1 [ CSSD]2013-05-23 17:18:50.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:50.409 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 336 (0x7f431bfc9810) [ CSSD]2013-05-23 17:18:50.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9810 (RPC#336) state 6, flags 0x202 [ CSSD]2013-05-23 17:18:50.409 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9810, ret 0, client (nil) [ CSSD]2013-05-23 17:18:50.409 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 336 [ CSSD]2013-05-23 17:18:51.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:51.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:51.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:53.195 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:53.428 [1191196992] >TRACE: clssgmAllocProc: (0x22e4cf0) allocated [ CSSD]2013-05-23 17:18:53.428 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:18:53.429 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e4cf0 - 1,2,3 [ CSSD]2013-05-23 17:18:53.429 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317e70) proc(0x22e4cf0) pid(20424) version 11:1:1:4 [ CSSD]2013-05-23 17:18:53.445 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e4cf0), client(1/0x2320f80) [ CSSD]2013-05-23 17:18:53.445 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2320f80) [ CSSD]2013-05-23 17:18:53.445 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2320f80 with con 0x22e4f70, requested num -1 [ CSSD]2013-05-23 17:18:53.445 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:53.445 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 337 (0x7f431bfc9890) [ CSSD]2013-05-23 17:18:53.445 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9890 (RPC#337) tag(151002d) sent to node 1 [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0151002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015152, gin 1, member 0 [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23210f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9890 (RPC#337) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9890, ret 0, client 0x2320f80 member 0x23185e0 [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2320f80 [ CSSD]2013-05-23 17:18:53.446 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 337 [ CSSD]2013-05-23 17:18:53.453 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2320f80) [ CSSD]2013-05-23 17:18:53.453 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2320f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:53.453 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23185e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:18:53.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 338 (0x7f431bfc9910) [ CSSD]2013-05-23 17:18:53.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9910 (RPC#338) tag(152002d) sent to node 1 [ CSSD]2013-05-23 17:18:53.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0152002d) incarnation(174631051) [ CSSD]2013-05-23 17:18:53.454 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:18:53.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9910 (RPC#338) state 6, flags 0x100 [ CSSD]2013-05-23 17:18:53.454 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9910, ret 0, client 0x2320f80 member 0x23185e0 [ CSSD]2013-05-23 17:18:53.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 338 [ CSSD]2013-05-23 17:18:53.454 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23185e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:18:53.656 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e4cf0 [ CSSD]2013-05-23 17:18:53.656 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e4cf0) con(0x2317e70) skgpid ospid 20424 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:18:53.785 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:53.785 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:18:53.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:53.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:53.842 [1191196992] >TRACE: clssgmAllocProc: (0x22e46f0) allocated [ CSSD]2013-05-23 17:18:53.843 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:18:53.844 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e46f0 - 1,2,3 [ CSSD]2013-05-23 17:18:53.844 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317d90) proc(0x22e46f0) pid(18636/18636) version 11:1:1:4 [ CSSD]2013-05-23 17:18:53.850 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22e46f0), client(1/0x22e4c30) [ CSSD]2013-05-23 17:18:53.850 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e4c30) [ CSSD]2013-05-23 17:18:53.850 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:18:53.850 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:18:54.745 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22ed180), client(2/0x2321180) [ CSSD]2013-05-23 17:18:54.745 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2321180) [ CSSD]2013-05-23 17:18:54.745 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:18:54.745 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 23 [ CSSD]2013-05-23 17:18:55.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:55.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:55.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:57.225 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:57.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:57.826 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:18:58.788 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:18:58.789 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:18:59.228 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:18:59.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:18:59.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:00.432 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:19:00.432 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 339 (0x7f431bfc9990) [ CSSD]2013-05-23 17:19:00.432 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9990 (RPC#339) tag(153002d) sent to node 1 [ CSSD]2013-05-23 17:19:00.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:00.432 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 339 (0x7f431bfc9990) [ CSSD]2013-05-23 17:19:00.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9990 (RPC#339) state 6, flags 0x202 [ CSSD]2013-05-23 17:19:00.432 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9990, ret 0, client (nil) [ CSSD]2013-05-23 17:19:00.433 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 339 [ CSSD]2013-05-23 17:19:01.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:01.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:01.886 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:03.256 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:03.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:03.292 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015153, gin 1, member 0 [ CSSD]2013-05-23 17:19:03.292 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:19:03.292 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:19:03.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:03.299 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:19:03.299 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:19:03.792 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:03.792 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:03.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:03.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:05.261 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:06.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:06.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:07.267 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:08.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:08.009 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:08.689 [1191196992] >TRACE: clssgmAllocProc: (0x22e7f20) allocated [ CSSD]2013-05-23 17:19:08.690 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:19:08.691 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e7f20 - 1,2,3 [ CSSD]2013-05-23 17:19:08.691 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23212f0) proc(0x22e7f20) pid(20575) version 11:1:1:4 [ CSSD]2013-05-23 17:19:08.696 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22e7f20), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:19:08.696 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:19:08.696 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:19:08.696 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:08.696 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 340 (0x7f431bfc9a10) [ CSSD]2013-05-23 17:19:08.697 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9a10 (RPC#340) tag(154002d) sent to node 1 [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0154002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015154, gin 1, member 0 [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9a10 (RPC#340) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9a10, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:19:08.697 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:19:08.698 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 340 [ CSSD]2013-05-23 17:19:08.703 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:19:08.704 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:08.704 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129c20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:08.704 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 341 (0x7f431bfc9a90) [ CSSD]2013-05-23 17:19:08.704 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9a90 (RPC#341) tag(155002d) sent to node 1 [ CSSD]2013-05-23 17:19:08.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0155002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:08.704 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:08.704 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9a90 (RPC#341) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:08.704 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9a90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:19:08.704 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 341 [ CSSD]2013-05-23 17:19:08.704 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129c20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:19:08.795 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:08.795 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:08.907 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e7f20 [ CSSD]2013-05-23 17:19:08.907 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e7f20) con(0x23212f0) skgpid ospid 20575 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:19:09.299 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:10.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:10.020 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:10.453 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:19:10.454 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 342 (0x7f431bfc9b10) [ CSSD]2013-05-23 17:19:10.454 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9b10 (RPC#342) tag(156002d) sent to node 1 [ CSSD]2013-05-23 17:19:10.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:10.454 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 342 (0x7f431bfc9b10) [ CSSD]2013-05-23 17:19:10.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9b10 (RPC#342) state 6, flags 0x202 [ CSSD]2013-05-23 17:19:10.454 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9b10, ret 0, client (nil) [ CSSD]2013-05-23 17:19:10.455 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 342 [ CSSD]2013-05-23 17:19:11.324 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:12.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:12.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:13.336 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:13.797 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:13.798 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:14.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:14.032 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:15.341 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:16.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:16.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:17.347 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:17.800 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:17.800 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:19:18.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:18.108 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:18.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:18.543 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015155, gin 1, member 0 [ CSSD]2013-05-23 17:19:18.543 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:19:18.543 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:19:18.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:18.550 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:19:18.550 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:19:19.399 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:20.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:20.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:20.486 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:19:20.486 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 343 (0x7f431bfc9b90) [ CSSD]2013-05-23 17:19:20.486 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9b90 (RPC#343) tag(157002d) sent to node 1 [ CSSD]2013-05-23 17:19:20.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:20.487 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 343 (0x7f431bfc9b90) [ CSSD]2013-05-23 17:19:20.487 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9b90 (RPC#343) state 6, flags 0x202 [ CSSD]2013-05-23 17:19:20.487 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9b90, ret 0, client (nil) [ CSSD]2013-05-23 17:19:20.487 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 343 [ CSSD]2013-05-23 17:19:21.405 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:22.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:22.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:22.802 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:22.802 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:23.417 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:23.939 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135050) allocated [ CSSD]2013-05-23 17:19:23.940 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:19:23.940 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135050 - 1,2,3 [ CSSD]2013-05-23 17:19:23.940 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c135050) pid(20728) version 11:1:1:4 [ CSSD]2013-05-23 17:19:23.946 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c135050), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:19:23.946 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:19:23.946 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:19:23.946 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:23.947 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 344 (0x7f431bfc9c10) [ CSSD]2013-05-23 17:19:23.947 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9c10 (RPC#344) tag(158002d) sent to node 1 [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0158002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015156, gin 1, member 0 [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9c10 (RPC#344) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9c10, ret 0, client 0x7f431c0e5910 member 0x7f431c10a490 [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:19:23.947 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 344 [ CSSD]2013-05-23 17:19:23.954 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:19:23.954 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:23.954 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:23.954 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 345 (0x7f431bfc9c90) [ CSSD]2013-05-23 17:19:23.955 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9c90 (RPC#345) tag(159002d) sent to node 1 [ CSSD]2013-05-23 17:19:23.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0159002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:23.955 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:23.955 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9c90 (RPC#345) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:23.955 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9c90, ret 0, client 0x7f431c0e5910 member 0x7f431c10a490 [ CSSD]2013-05-23 17:19:23.955 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 345 [ CSSD]2013-05-23 17:19:23.955 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:19:24.157 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135050 [ CSSD]2013-05-23 17:19:24.157 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135050) con(0x7f431c12a550) skgpid ospid 20728 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:19:24.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:24.173 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:25.423 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:26.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:26.184 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:27.429 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:27.806 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:27.806 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:28.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:28.191 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:29.235 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:19:29.235 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:19:29.237 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:19:29.237 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:19:29.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:29.314 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015157, gin 1, member 0 [ CSSD]2013-05-23 17:19:29.314 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c13edd0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:19:29.314 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:19:29.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:29.321 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:19:29.321 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:19:29.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:30.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:30.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:30.508 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:19:30.509 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 346 (0x7f431bfc9d10) [ CSSD]2013-05-23 17:19:30.509 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9d10 (RPC#346) tag(15a002d) sent to node 1 [ CSSD]2013-05-23 17:19:30.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:30.509 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 346 (0x7f431bfc9d10) [ CSSD]2013-05-23 17:19:30.509 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9d10 (RPC#346) state 6, flags 0x202 [ CSSD]2013-05-23 17:19:30.509 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9d10, ret 0, client (nil) [ CSSD]2013-05-23 17:19:30.510 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 346 [ CSSD]2013-05-23 17:19:30.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:30.696 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015158, gin 1, member 0 [ CSSD]2013-05-23 17:19:30.696 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:19:30.696 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:19:30.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:30.703 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:19:30.703 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:19:31.464 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:32.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:32.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:32.809 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:32.809 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:33.497 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:33.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:33.800 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015159, gin 1, member 0 [ CSSD]2013-05-23 17:19:33.800 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:19:33.800 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:19:33.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:33.807 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:19:33.807 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:19:34.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:34.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:34.569 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e6d60) allocated [ CSSD]2013-05-23 17:19:34.570 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:19:34.571 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e6d60 - 1,2,3 [ CSSD]2013-05-23 17:19:34.571 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c0e6d60) pid(20843) version 11:1:1:4 [ CSSD]2013-05-23 17:19:34.577 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c0e6d60), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:19:34.577 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:19:34.577 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:19:34.577 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:34.577 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 347 (0x7f431bfc9d90) [ CSSD]2013-05-23 17:19:34.577 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9d90 (RPC#347) tag(15b002d) sent to node 1 [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(015b002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015160, gin 1, member 0 [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9d90 (RPC#347) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9d90, ret 0, client 0x7f431c0cf720 member 0x7f431c10a490 [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:19:34.578 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 347 [ CSSD]2013-05-23 17:19:34.584 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:19:34.584 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:34.584 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:34.584 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 348 (0x7f431bfc9e10) [ CSSD]2013-05-23 17:19:34.584 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9e10 (RPC#348) tag(15c002d) sent to node 1 [ CSSD]2013-05-23 17:19:34.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(015c002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:34.585 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:34.585 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9e10 (RPC#348) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:34.585 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9e10, ret 0, client 0x7f431c0cf720 member 0x7f431c10a490 [ CSSD]2013-05-23 17:19:34.585 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 348 [ CSSD]2013-05-23 17:19:34.585 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:19:34.787 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e6d60 [ CSSD]2013-05-23 17:19:34.787 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e6d60) con(0x7f431c12a550) skgpid ospid 20843 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:19:35.502 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:36.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:36.245 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:37.077 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:19:37.078 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:19:37.078 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:19:37.078 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c10a490) pid(20478/20478) version 11:1:1:4 [ CSSD]2013-05-23 17:19:37.084 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a490), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:19:37.084 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:19:37.084 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:19:37.084 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:19:37.526 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:37.536 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c10a490), client(2/0x7f431c12e6d0) [ CSSD]2013-05-23 17:19:37.536 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:19:37.536 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:19:37.536 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 17:19:37.811 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:37.812 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:38.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:38.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:39.198 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13f950) allocated [ CSSD]2013-05-23 17:19:39.199 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:19:39.200 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13f950 - 1,2,3 [ CSSD]2013-05-23 17:19:39.200 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c13f950) pid(20876) version 11:1:1:4 [ CSSD]2013-05-23 17:19:39.206 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13f950), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:19:39.206 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:19:39.206 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:19:39.206 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:39.206 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 349 (0x7f431bfc9e90) [ CSSD]2013-05-23 17:19:39.206 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9e90 (RPC#349) tag(15d002d) sent to node 1 [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(015d002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015161, gin 1, member 0 [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9e90 (RPC#349) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfc9e90, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:19:39.207 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 349 [ CSSD]2013-05-23 17:19:39.213 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:19:39.213 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:39.213 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:39.213 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 350 (0x7f431bfc9f10) [ CSSD]2013-05-23 17:19:39.213 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9f10 (RPC#350) tag(15e002d) sent to node 1 [ CSSD]2013-05-23 17:19:39.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(015e002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:39.214 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:39.214 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9f10 (RPC#350) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:39.214 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfc9f10, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 17:19:39.214 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 350 [ CSSD]2013-05-23 17:19:39.214 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:19:39.416 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13f950 [ CSSD]2013-05-23 17:19:39.416 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13f950) con(0x7f431c108e60) skgpid ospid 20876 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:19:39.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:40.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:40.302 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:40.532 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:19:40.532 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 351 (0x7f431bfc9f90) [ CSSD]2013-05-23 17:19:40.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfc9f90 (RPC#351) tag(15f002d) sent to node 1 [ CSSD]2013-05-23 17:19:40.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:40.533 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 351 (0x7f431bfc9f90) [ CSSD]2013-05-23 17:19:40.533 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfc9f90 (RPC#351) state 6, flags 0x202 [ CSSD]2013-05-23 17:19:40.533 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfc9f90, ret 0, client (nil) [ CSSD]2013-05-23 17:19:40.533 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 351 [ CSSD]2013-05-23 17:19:41.563 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:41.814 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:41.814 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:19:42.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:42.329 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:43.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:44.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:44.336 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:45.603 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:46.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:46.344 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:46.816 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:46.817 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:47.627 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:48.350 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:48.350 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:49.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:49.055 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015162, gin 1, member 0 [ CSSD]2013-05-23 17:19:49.055 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:19:49.055 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:19:49.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:49.062 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:19:49.062 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:19:49.658 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:50.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:50.403 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:50.557 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:19:50.557 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 352 (0x7f431bfca010) [ CSSD]2013-05-23 17:19:50.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca010 (RPC#352) tag(160002d) sent to node 1 [ CSSD]2013-05-23 17:19:50.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:50.558 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 352 (0x7f431bfca010) [ CSSD]2013-05-23 17:19:50.558 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca010 (RPC#352) state 6, flags 0x202 [ CSSD]2013-05-23 17:19:50.558 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca010, ret 0, client (nil) [ CSSD]2013-05-23 17:19:50.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 352 [ CSSD]2013-05-23 17:19:51.664 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:51.819 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:51.819 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:52.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:52.410 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:53.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:54.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:54.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:54.449 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 17:19:54.451 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:19:54.451 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 17:19:54.451 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c13ea10) pid(21020) version 11:1:1:4 [ CSSD]2013-05-23 17:19:54.457 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13ea10), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:19:54.457 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:19:54.457 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:19:54.457 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:54.457 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 353 (0x7f431bfca090) [ CSSD]2013-05-23 17:19:54.457 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca090 (RPC#353) tag(161002d) sent to node 1 [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0161002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015163, gin 1, member 0 [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca090 (RPC#353) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca090, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:19:54.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 353 [ CSSD]2013-05-23 17:19:54.466 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:19:54.466 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:54.466 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:19:54.466 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 354 (0x7f431bfca110) [ CSSD]2013-05-23 17:19:54.466 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca110 (RPC#354) tag(162002d) sent to node 1 [ CSSD]2013-05-23 17:19:54.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0162002d) incarnation(174631051) [ CSSD]2013-05-23 17:19:54.467 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:19:54.467 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca110 (RPC#354) state 6, flags 0x100 [ CSSD]2013-05-23 17:19:54.467 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfca110, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:19:54.467 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 354 [ CSSD]2013-05-23 17:19:54.467 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:19:54.669 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 17:19:54.669 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c12f0c0) skgpid ospid 21020 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:19:55.704 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:56.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:56.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:56.822 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:19:56.823 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:19:57.710 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:19:58.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:19:58.453 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:19:59.722 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:00.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:00.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:00.580 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:20:00.581 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 355 (0x7f431bfca190) [ CSSD]2013-05-23 17:20:00.581 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca190 (RPC#355) tag(163002d) sent to node 1 [ CSSD]2013-05-23 17:20:00.581 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:00.581 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 355 (0x7f431bfca190) [ CSSD]2013-05-23 17:20:00.581 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca190 (RPC#355) state 6, flags 0x202 [ CSSD]2013-05-23 17:20:00.581 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca190, ret 0, client (nil) [ CSSD]2013-05-23 17:20:00.581 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 355 [ CSSD]2013-05-23 17:20:00.825 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:00.825 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:20:01.727 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:02.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:02.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:03.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:04.305 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:04.305 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015164, gin 1, member 0 [ CSSD]2013-05-23 17:20:04.305 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:20:04.305 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:20:04.312 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:04.312 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:20:04.312 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:20:04.521 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:04.521 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:05.823 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:05.828 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:05.828 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:06.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:06.528 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:07.828 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:07.972 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 17:20:07.972 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:20:07.973 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 17:20:07.973 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c108370) pid(21350) version 11:1:1:4 [ CSSD]2013-05-23 17:20:08.532 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 17:20:08.532 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11ab50) skgpid ospid 21350 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:08.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:08.554 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:09.701 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13f950) allocated [ CSSD]2013-05-23 17:20:09.702 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:20:09.702 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13f950 - 1,2,3 [ CSSD]2013-05-23 17:20:09.702 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c13f950) pid(21458) version 11:1:1:4 [ CSSD]2013-05-23 17:20:09.709 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13f950), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:20:09.709 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:20:09.709 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:20:09.709 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:09.709 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 356 (0x7f431bfca210) [ CSSD]2013-05-23 17:20:09.709 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca210 (RPC#356) tag(164002d) sent to node 1 [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0164002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015165, gin 1, member 0 [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca210 (RPC#356) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca210, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:20:09.710 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 356 [ CSSD]2013-05-23 17:20:09.718 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:20:09.718 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:09.718 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:09.718 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 357 (0x7f431bfca290) [ CSSD]2013-05-23 17:20:09.718 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca290 (RPC#357) tag(165002d) sent to node 1 [ CSSD]2013-05-23 17:20:09.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0165002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:09.718 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:09.718 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca290 (RPC#357) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:09.718 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfca290, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:09.718 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 357 [ CSSD]2013-05-23 17:20:09.718 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:20:09.840 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:09.921 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13f950 [ CSSD]2013-05-23 17:20:09.921 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13f950) con(0x7f431c12a550) skgpid ospid 21458 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:10.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:10.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:10.605 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:20:10.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 358 (0x7f431bfca310) [ CSSD]2013-05-23 17:20:10.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca310 (RPC#358) tag(166002d) sent to node 1 [ CSSD]2013-05-23 17:20:10.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:10.606 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 358 (0x7f431bfca310) [ CSSD]2013-05-23 17:20:10.606 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca310 (RPC#358) state 6, flags 0x202 [ CSSD]2013-05-23 17:20:10.606 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca310, ret 0, client (nil) [ CSSD]2013-05-23 17:20:10.606 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 358 [ CSSD]2013-05-23 17:20:10.830 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:10.830 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:11.845 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:12.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:12.611 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:13.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:14.701 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:14.701 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:15.833 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:15.833 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:15.924 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:16.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:16.727 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:17.930 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:18.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:18.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:19.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:19.556 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015166, gin 1, member 0 [ CSSD]2013-05-23 17:20:19.556 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:20:19.556 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:20:19.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:19.564 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:20:19.564 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:20:19.942 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:20.110 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:20:20.110 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2312940), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:20:20.110 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2312940), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:20:20.110 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:20:20.110 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e2eb0 [ CSSD]2013-05-23 17:20:20.110 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e2eb0) con(0x2302d20) skgpid 16338 ospid 16338 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:20.628 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:20:20.628 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 359 (0x7f431bfca390) [ CSSD]2013-05-23 17:20:20.629 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca390 (RPC#359) tag(167002d) sent to node 1 [ CSSD]2013-05-23 17:20:20.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:20.629 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 359 (0x7f431bfca390) [ CSSD]2013-05-23 17:20:20.629 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca390 (RPC#359) state 6, flags 0x202 [ CSSD]2013-05-23 17:20:20.629 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca390, ret 0, client (nil) [ CSSD]2013-05-23 17:20:20.629 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 359 [ CSSD]2013-05-23 17:20:20.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:20.743 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:20.837 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:20.837 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:21.948 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:22.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:22.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:23.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:24.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:24.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:24.839 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:24.839 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:20:24.953 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 17:20:24.954 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:20:24.954 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 17:20:24.954 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c13ea10) pid(21636) version 11:1:1:4 [ CSSD]2013-05-23 17:20:24.961 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ea10), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:20:24.961 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:20:24.961 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:20:24.961 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:24.961 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 360 (0x7f431bfca410) [ CSSD]2013-05-23 17:20:24.961 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca410 (RPC#360) tag(168002d) sent to node 1 [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0168002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015167, gin 1, member 0 [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca410 (RPC#360) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca410, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:20:24.962 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 360 [ CSSD]2013-05-23 17:20:24.968 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:20:24.968 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:24.968 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:24.968 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 361 (0x7f431bfca490) [ CSSD]2013-05-23 17:20:24.968 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca490 (RPC#361) tag(169002d) sent to node 1 [ CSSD]2013-05-23 17:20:24.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0169002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:24.968 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:24.968 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca490 (RPC#361) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:24.968 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfca490, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:24.968 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 361 [ CSSD]2013-05-23 17:20:24.968 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:20:25.171 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 17:20:25.171 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c12a550) skgpid ospid 21636 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:25.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:26.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:26.808 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:28.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:28.834 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:28.834 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:29.843 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:29.843 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:30.013 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:30.656 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:20:30.656 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 362 (0x7f431bfca510) [ CSSD]2013-05-23 17:20:30.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca510 (RPC#362) tag(16a002d) sent to node 1 [ CSSD]2013-05-23 17:20:30.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:30.657 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 362 (0x7f431bfca510) [ CSSD]2013-05-23 17:20:30.657 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca510 (RPC#362) state 6, flags 0x202 [ CSSD]2013-05-23 17:20:30.657 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca510, ret 0, client (nil) [ CSSD]2013-05-23 17:20:30.657 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 362 [ CSSD]2013-05-23 17:20:30.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:30.844 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:30.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:30.949 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015168, gin 1, member 0 [ CSSD]2013-05-23 17:20:30.949 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:20:30.949 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:20:30.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:30.956 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:20:30.956 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:20:32.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:32.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:32.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:34.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:34.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:34.805 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015169, gin 1, member 0 [ CSSD]2013-05-23 17:20:34.805 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:20:34.805 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:20:34.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:34.810 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:20:34.810 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c048500) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:20:34.822 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13f950) allocated [ CSSD]2013-05-23 17:20:34.823 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:20:34.824 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13f950 - 1,2,3 [ CSSD]2013-05-23 17:20:34.824 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c13f950) pid(21741) version 11:1:1:4 [ CSSD]2013-05-23 17:20:34.840 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13f950), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:20:34.841 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:20:34.841 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:20:34.841 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:34.841 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 363 (0x7f431bfca590) [ CSSD]2013-05-23 17:20:34.841 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca590 (RPC#363) tag(16b002d) sent to node 1 [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(016b002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015170, gin 1, member 0 [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca590 (RPC#363) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca590, ret 0, client 0x7f431c0e5910 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:20:34.841 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 363 [ CSSD]2013-05-23 17:20:34.845 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:34.846 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:34.848 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:20:34.848 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:34.848 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:34.848 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 364 (0x7f431bfca610) [ CSSD]2013-05-23 17:20:34.848 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca610 (RPC#364) tag(16c002d) sent to node 1 [ CSSD]2013-05-23 17:20:34.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(016c002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:34.849 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:34.849 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca610 (RPC#364) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:34.849 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfca610, ret 0, client 0x7f431c0e5910 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:34.849 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 364 [ CSSD]2013-05-23 17:20:34.849 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:20:34.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:34.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:35.052 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13f950 [ CSSD]2013-05-23 17:20:35.052 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13f950) con(0x7f431c12a550) skgpid ospid 21741 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:36.055 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:36.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:36.908 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:38.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:38.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:38.934 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:39.849 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:39.849 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:40.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:40.203 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 17:20:40.204 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:20:40.205 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 17:20:40.205 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c108370) pid(21806) version 11:1:1:4 [ CSSD]2013-05-23 17:20:40.221 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c108370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:20:40.221 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:20:40.221 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:20:40.221 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:40.221 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 365 (0x7f431bfca690) [ CSSD]2013-05-23 17:20:40.221 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca690 (RPC#365) tag(16d002d) sent to node 1 [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(016d002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015171, gin 1, member 0 [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca690 (RPC#365) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca690, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:20:40.222 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 365 [ CSSD]2013-05-23 17:20:40.228 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:20:40.228 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:40.228 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:40.228 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 366 (0x7f431bfca710) [ CSSD]2013-05-23 17:20:40.228 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca710 (RPC#366) tag(16e002d) sent to node 1 [ CSSD]2013-05-23 17:20:40.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(016e002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:40.228 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:40.228 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca710 (RPC#366) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:40.228 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfca710, ret 0, client 0x7f431c109f80 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:40.228 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 366 [ CSSD]2013-05-23 17:20:40.228 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:20:40.431 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 17:20:40.431 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c12f0c0) skgpid ospid 21806 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:40.679 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:20:40.679 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 367 (0x7f431bfca790) [ CSSD]2013-05-23 17:20:40.680 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca790 (RPC#367) tag(16f002d) sent to node 1 [ CSSD]2013-05-23 17:20:40.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:40.680 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 367 (0x7f431bfca790) [ CSSD]2013-05-23 17:20:40.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca790 (RPC#367) state 6, flags 0x202 [ CSSD]2013-05-23 17:20:40.680 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca790, ret 0, client (nil) [ CSSD]2013-05-23 17:20:40.680 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 367 [ CSSD]2013-05-23 17:20:40.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:40.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:42.120 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:42.972 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:42.972 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:44.150 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:44.852 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:44.852 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:45.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:45.003 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:46.174 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:47.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:47.010 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:48.208 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:49.037 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:49.037 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:49.855 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:49.855 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:50.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:50.059 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015172, gin 1, member 0 [ CSSD]2013-05-23 17:20:50.059 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:20:50.059 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:20:50.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:50.066 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:20:50.066 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c12f520) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:20:50.213 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:50.704 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:20:50.705 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 368 (0x7f431bfca810) [ CSSD]2013-05-23 17:20:50.705 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca810 (RPC#368) tag(170002d) sent to node 1 [ CSSD]2013-05-23 17:20:50.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:50.705 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 368 (0x7f431bfca810) [ CSSD]2013-05-23 17:20:50.705 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca810 (RPC#368) state 6, flags 0x202 [ CSSD]2013-05-23 17:20:50.705 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfca810, ret 0, client (nil) [ CSSD]2013-05-23 17:20:50.705 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 368 [ CSSD]2013-05-23 17:20:51.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:51.047 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:52.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:53.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:53.054 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:54.231 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:54.858 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:54.858 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:20:55.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:55.074 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:55.462 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 17:20:55.464 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:20:55.464 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 17:20:55.464 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c13ea10) pid(21947) version 11:1:1:4 [ CSSD]2013-05-23 17:20:55.470 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13ea10), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:20:55.470 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:20:55.470 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:20:55.470 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:55.470 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 369 (0x7f431bfca890) [ CSSD]2013-05-23 17:20:55.470 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca890 (RPC#369) tag(171002d) sent to node 1 [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0171002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015173, gin 1, member 0 [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca890 (RPC#369) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca890, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:55.471 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:20:55.472 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 369 [ CSSD]2013-05-23 17:20:55.479 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:20:55.479 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:55.479 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:55.479 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 370 (0x7f431bfca910) [ CSSD]2013-05-23 17:20:55.479 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca910 (RPC#370) tag(172002d) sent to node 1 [ CSSD]2013-05-23 17:20:55.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0172002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:55.479 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:55.479 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca910 (RPC#370) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:55.479 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfca910, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:55.480 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 370 [ CSSD]2013-05-23 17:20:55.480 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:20:55.682 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 17:20:55.682 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c12f0c0) skgpid ospid 21947 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:56.237 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:57.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:57.081 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:58.309 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:20:58.713 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 17:20:58.715 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:20:58.715 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 17:20:58.715 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c108370) pid(22010) version 11:1:1:4 [ CSSD]2013-05-23 17:20:58.721 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:20:58.721 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:20:58.721 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:20:58.721 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:58.721 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 371 (0x7f431bfca990) [ CSSD]2013-05-23 17:20:58.721 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfca990 (RPC#371) tag(173002d) sent to node 1 [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0173002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015174, gin 1, member 0 [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfca990 (RPC#371) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfca990, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:58.722 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:20:58.723 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 371 [ CSSD]2013-05-23 17:20:58.729 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:20:58.729 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:58.729 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095660) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:20:58.729 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 372 (0x7f431bfcaa10) [ CSSD]2013-05-23 17:20:58.729 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcaa10 (RPC#372) tag(174002d) sent to node 1 [ CSSD]2013-05-23 17:20:58.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0174002d) incarnation(174631051) [ CSSD]2013-05-23 17:20:58.729 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:20:58.729 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcaa10 (RPC#372) state 6, flags 0x100 [ CSSD]2013-05-23 17:20:58.729 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcaa10, ret 0, client 0x7f431c0cf720 member 0x7f431c095660 [ CSSD]2013-05-23 17:20:58.730 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 372 [ CSSD]2013-05-23 17:20:58.730 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095660) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:20:58.932 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 17:20:58.932 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c108e60) skgpid ospid 22010 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:20:59.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:20:59.087 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:20:59.861 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:20:59.862 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:00.333 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:00.733 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:21:00.733 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 373 (0x7f431bfcaa90) [ CSSD]2013-05-23 17:21:00.733 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcaa90 (RPC#373) tag(175002d) sent to node 1 [ CSSD]2013-05-23 17:21:00.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:00.734 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 373 (0x7f431bfcaa90) [ CSSD]2013-05-23 17:21:00.734 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcaa90 (RPC#373) state 6, flags 0x202 [ CSSD]2013-05-23 17:21:00.734 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcaa90, ret 0, client (nil) [ CSSD]2013-05-23 17:21:00.734 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 373 [ CSSD]2013-05-23 17:21:01.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:01.117 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:02.339 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:03.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:03.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:04.352 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:04.865 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:04.865 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:05.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:05.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:05.201 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13f950) allocated [ CSSD]2013-05-23 17:21:05.202 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:21:05.202 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13f950 - 1,2,3 [ CSSD]2013-05-23 17:21:05.202 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c13f950) pid(21601/21601) version 11:1:1:4 [ CSSD]2013-05-23 17:21:05.218 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13f950), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:21:05.218 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:21:05.218 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:21:05.218 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:21:05.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:05.335 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015175, gin 1, member 0 [ CSSD]2013-05-23 17:21:05.335 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:21:05.335 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:21:05.342 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:05.342 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:21:05.342 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c095690) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:21:06.358 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:07.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:07.159 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:08.409 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:09.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:09.164 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:09.868 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:09.868 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:10.416 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:10.724 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:21:10.724 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:21:10.725 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:21:10.725 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c12b920) pid(22113) version 11:1:1:4 [ CSSD]2013-05-23 17:21:10.731 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:21:10.731 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:21:10.731 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:21:10.731 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:10.731 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 374 (0x7f431bfcab10) [ CSSD]2013-05-23 17:21:10.731 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcab10 (RPC#374) tag(176002d) sent to node 1 [ CSSD]2013-05-23 17:21:10.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0176002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:10.731 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015176, gin 1, member 0 [ CSSD]2013-05-23 17:21:10.731 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:21:10.732 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:21:10.732 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:21:10.732 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcab10 (RPC#374) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:10.732 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcab10, ret 0, client 0x7f431c0cf720 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:10.732 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:21:10.732 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 374 [ CSSD]2013-05-23 17:21:10.738 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:21:10.738 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:10.738 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eec0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:10.738 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 375 (0x7f431bfcab90) [ CSSD]2013-05-23 17:21:10.738 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcab90 (RPC#375) tag(177002d) sent to node 1 [ CSSD]2013-05-23 17:21:10.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0177002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:10.738 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:10.738 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcab90 (RPC#375) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:10.738 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcab90, ret 0, client 0x7f431c0cf720 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:10.739 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 375 [ CSSD]2013-05-23 17:21:10.739 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eec0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:21:10.758 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:21:10.758 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 376 (0x7f431bfcac10) [ CSSD]2013-05-23 17:21:10.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcac10 (RPC#376) tag(178002d) sent to node 1 [ CSSD]2013-05-23 17:21:10.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:10.759 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 376 (0x7f431bfcac10) [ CSSD]2013-05-23 17:21:10.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcac10 (RPC#376) state 6, flags 0x202 [ CSSD]2013-05-23 17:21:10.759 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcac10, ret 0, client (nil) [ CSSD]2013-05-23 17:21:10.759 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 376 [ CSSD]2013-05-23 17:21:10.941 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:21:10.941 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c11ab50) skgpid ospid 22113 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:11.217 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:11.217 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:12.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:13.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:13.224 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:14.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:14.871 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:14.871 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:15.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:15.235 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:16.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:17.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:17.242 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:18.510 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:19.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:19.267 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:19.875 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:19.875 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:20.516 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:20.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:20.592 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015177, gin 1, member 0 [ CSSD]2013-05-23 17:21:20.592 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:21:20.592 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:21:20.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:20.599 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:21:20.599 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c095690) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:21:20.782 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:21:20.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 377 (0x7f431bfcac90) [ CSSD]2013-05-23 17:21:20.782 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcac90 (RPC#377) tag(179002d) sent to node 1 [ CSSD]2013-05-23 17:21:20.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:20.783 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 377 (0x7f431bfcac90) [ CSSD]2013-05-23 17:21:20.783 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcac90 (RPC#377) state 6, flags 0x202 [ CSSD]2013-05-23 17:21:20.783 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcac90, ret 0, client (nil) [ CSSD]2013-05-23 17:21:20.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 377 [ CSSD]2013-05-23 17:21:20.902 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:21:20.903 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:21:20.904 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:21:20.904 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c12b920) pid(22226) version 11:1:1:4 [ CSSD]2013-05-23 17:21:20.910 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:21:20.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:21:20.910 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:21:20.910 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:21:20.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 378 (0x7f431bfcad10) [ CSSD]2013-05-23 17:21:20.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcad10 (RPC#378) tag(17a002d) sent to node 1 [ CSSD]2013-05-23 17:21:20.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(017a002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:20.910 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015178, gin 1, member 0 [ CSSD]2013-05-23 17:21:20.910 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:21:20.911 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:21:20.911 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:21:20.911 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcad10 (RPC#378) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:20.911 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcad10, ret 0, client 0x7f431c0cf720 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:20.911 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:21:20.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 378 [ CSSD]2013-05-23 17:21:20.917 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:21:20.917 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:21:20.917 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eec0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:21:20.917 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 379 (0x7f431bfcad90) [ CSSD]2013-05-23 17:21:20.917 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcad90 (RPC#379) tag(17b002d) sent to node 1 [ CSSD]2013-05-23 17:21:20.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(017b002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:20.917 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:21:20.917 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcad90 (RPC#379) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:20.917 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcad90, ret 0, client 0x7f431c0cf720 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:20.917 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 379 [ CSSD]2013-05-23 17:21:20.917 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c13eec0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:21:21.028 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136070) allocated [ CSSD]2013-05-23 17:21:21.029 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:21:21.029 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136070 - 1,2,3 [ CSSD]2013-05-23 17:21:21.029 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c136070) pid(22231) version 11:1:1:4 [ CSSD]2013-05-23 17:21:21.043 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:21:21.045 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:21:21.048 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136070 [ CSSD]2013-05-23 17:21:21.048 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136070) con(0x7f431c11ab50) skgpid ospid 22231 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:21.120 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:21:21.120 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c12f0c0) skgpid ospid 22226 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:21.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:21.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:22.542 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:23.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:23.325 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:24.552 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:24.878 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:24.878 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:25.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:25.335 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:25.978 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:21:25.979 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:21:25.980 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:21:25.980 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c12b920) pid(22279) version 11:1:1:4 [ CSSD]2013-05-23 17:21:25.986 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:21:25.986 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:25.986 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:21:25.986 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:25.986 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 380 (0x7f431bfcae10) [ CSSD]2013-05-23 17:21:25.986 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcae10 (RPC#380) tag(17c002d) sent to node 1 [ CSSD]2013-05-23 17:21:25.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(017c002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:25.986 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015179, gin 1, member 0 [ CSSD]2013-05-23 17:21:25.986 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:21:25.986 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:21:25.986 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:21:25.987 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcae10 (RPC#380) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:25.987 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcae10, ret 0, client 0x7f431c109f80 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:25.987 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:21:25.987 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 380 [ CSSD]2013-05-23 17:21:25.993 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:25.993 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:25.993 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eec0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:25.993 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 381 (0x7f431bfcae90) [ CSSD]2013-05-23 17:21:25.993 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcae90 (RPC#381) tag(17d002d) sent to node 1 [ CSSD]2013-05-23 17:21:25.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(017d002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:25.994 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:25.994 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcae90 (RPC#381) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:25.994 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcae90, ret 0, client 0x7f431c109f80 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:25.994 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 381 [ CSSD]2013-05-23 17:21:25.994 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eec0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:21:26.196 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:21:26.197 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c11ab50) skgpid ospid 22279 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:26.558 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:27.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:27.341 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:28.610 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:29.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:29.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:29.881 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:29.882 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:30.616 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:30.806 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:21:30.807 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 382 (0x7f431bfcaf10) [ CSSD]2013-05-23 17:21:30.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcaf10 (RPC#382) tag(17e002d) sent to node 1 [ CSSD]2013-05-23 17:21:30.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:30.808 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 382 (0x7f431bfcaf10) [ CSSD]2013-05-23 17:21:30.808 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcaf10 (RPC#382) state 6, flags 0x202 [ CSSD]2013-05-23 17:21:30.808 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcaf10, ret 0, client (nil) [ CSSD]2013-05-23 17:21:30.808 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 382 [ CSSD]2013-05-23 17:21:31.216 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:31.216 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015180, gin 1, member 0 [ CSSD]2013-05-23 17:21:31.216 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:21:31.216 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:21:31.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:31.223 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:21:31.223 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c095670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:21:31.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:31.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:32.622 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:33.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:33.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:34.632 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:34.885 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:34.885 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:35.088 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:21:35.090 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:21:35.090 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:21:35.090 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c12b920) pid(22396) version 11:1:1:4 [ CSSD]2013-05-23 17:21:35.097 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:21:35.097 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:35.097 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:21:35.097 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:35.097 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 383 (0x7f431bfcaf90) [ CSSD]2013-05-23 17:21:35.097 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcaf90 (RPC#383) tag(17f002d) sent to node 1 [ CSSD]2013-05-23 17:21:35.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(017f002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:35.097 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015181, gin 1, member 0 [ CSSD]2013-05-23 17:21:35.097 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:21:35.098 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:21:35.098 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:21:35.098 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcaf90 (RPC#383) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:35.098 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcaf90, ret 0, client 0x7f431c109f80 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:35.098 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:21:35.098 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 383 [ CSSD]2013-05-23 17:21:35.104 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:35.104 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:35.104 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13eec0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:35.104 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 384 (0x7f431bfcb010) [ CSSD]2013-05-23 17:21:35.104 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb010 (RPC#384) tag(180002d) sent to node 1 [ CSSD]2013-05-23 17:21:35.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0180002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:35.105 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:35.105 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb010 (RPC#384) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:35.105 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb010, ret 0, client 0x7f431c109f80 member 0x7f431c13eec0 [ CSSD]2013-05-23 17:21:35.105 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 384 [ CSSD]2013-05-23 17:21:35.105 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13eec0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:21:35.307 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:21:35.307 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c11ab50) skgpid ospid 22396 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:35.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:35.455 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:35.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:35.842 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015182, gin 1, member 0 [ CSSD]2013-05-23 17:21:35.842 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:21:35.842 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:21:35.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:35.849 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:21:35.849 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c095690) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:21:36.659 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:37.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:37.461 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:38.661 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:39.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:39.513 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:39.888 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:39.888 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:40.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:40.832 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:21:40.832 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 385 (0x7f431bfcb090) [ CSSD]2013-05-23 17:21:40.832 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb090 (RPC#385) tag(181002d) sent to node 1 [ CSSD]2013-05-23 17:21:40.833 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:40.833 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 385 (0x7f431bfcb090) [ CSSD]2013-05-23 17:21:40.833 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb090 (RPC#385) state 6, flags 0x202 [ CSSD]2013-05-23 17:21:40.833 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcb090, ret 0, client (nil) [ CSSD]2013-05-23 17:21:40.833 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 385 [ CSSD]2013-05-23 17:21:41.229 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:21:41.231 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:21:41.231 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:21:41.231 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c12b920) pid(22471) version 11:1:1:4 [ CSSD]2013-05-23 17:21:41.237 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:21:41.237 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:41.237 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:21:41.238 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:41.238 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 386 (0x7f431bfcb110) [ CSSD]2013-05-23 17:21:41.238 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb110 (RPC#386) tag(182002d) sent to node 1 [ CSSD]2013-05-23 17:21:41.238 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0182002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:41.238 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015183, gin 1, member 0 [ CSSD]2013-05-23 17:21:41.238 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:21:41.239 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:21:41.239 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:21:41.239 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb110 (RPC#386) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:41.239 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb110, ret 0, client 0x7f431c109f80 member 0x7f431c1216c0 [ CSSD]2013-05-23 17:21:41.239 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:21:41.239 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 386 [ CSSD]2013-05-23 17:21:41.245 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:41.245 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:41.245 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1216c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:41.245 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 387 (0x7f431bfcb190) [ CSSD]2013-05-23 17:21:41.245 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb190 (RPC#387) tag(183002d) sent to node 1 [ CSSD]2013-05-23 17:21:41.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0183002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:41.246 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:41.246 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb190 (RPC#387) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:41.246 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb190, ret 0, client 0x7f431c109f80 member 0x7f431c1216c0 [ CSSD]2013-05-23 17:21:41.246 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 387 [ CSSD]2013-05-23 17:21:41.246 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1216c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:21:41.448 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:21:41.448 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c12f0c0) skgpid ospid 22471 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:41.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:41.520 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:42.693 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:43.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:43.527 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:43.891 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:43.891 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:21:44.744 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:45.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:45.555 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:46.750 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:47.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:47.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:48.761 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:48.894 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:48.894 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:49.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:49.614 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:50.767 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:50.856 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:21:50.856 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 388 (0x7f431bfcb210) [ CSSD]2013-05-23 17:21:50.856 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb210 (RPC#388) tag(184002d) sent to node 1 [ CSSD]2013-05-23 17:21:50.857 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:50.857 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 388 (0x7f431bfcb210) [ CSSD]2013-05-23 17:21:50.857 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb210 (RPC#388) state 6, flags 0x202 [ CSSD]2013-05-23 17:21:50.857 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcb210, ret 0, client (nil) [ CSSD]2013-05-23 17:21:50.857 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 388 [ CSSD]2013-05-23 17:21:51.103 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:51.103 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015184, gin 1, member 0 [ CSSD]2013-05-23 17:21:51.103 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:21:51.103 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:21:51.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:51.112 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:21:51.112 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c095670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:21:51.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:51.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:52.791 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:53.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:53.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:53.897 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:53.897 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:54.844 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:55.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:55.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:56.490 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:21:56.491 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:21:56.492 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:21:56.492 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c12b920) pid(22617) version 11:1:1:4 [ CSSD]2013-05-23 17:21:56.498 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:21:56.498 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:56.498 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:21:56.498 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:56.498 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 389 (0x7f431bfcb290) [ CSSD]2013-05-23 17:21:56.498 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb290 (RPC#389) tag(185002d) sent to node 1 [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0185002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015185, gin 1, member 0 [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb290 (RPC#389) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb290, ret 0, client 0x7f431c109f80 member 0x7f431c1216c0 [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:21:56.499 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 389 [ CSSD]2013-05-23 17:21:56.505 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:21:56.505 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:56.505 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1216c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:21:56.505 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 390 (0x7f431bfcb310) [ CSSD]2013-05-23 17:21:56.505 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb310 (RPC#390) tag(186002d) sent to node 1 [ CSSD]2013-05-23 17:21:56.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0186002d) incarnation(174631051) [ CSSD]2013-05-23 17:21:56.506 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:21:56.506 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb310 (RPC#390) state 6, flags 0x100 [ CSSD]2013-05-23 17:21:56.506 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb310, ret 0, client 0x7f431c109f80 member 0x7f431c1216c0 [ CSSD]2013-05-23 17:21:56.506 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 390 [ CSSD]2013-05-23 17:21:56.506 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1216c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:21:56.709 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:21:56.709 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c108e60) skgpid ospid 22617 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:21:56.850 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:57.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:57.663 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:21:58.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:21:58.900 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:21:58.900 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:21:59.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:21:59.714 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:00.867 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:00.881 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:22:00.881 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 391 (0x7f431bfcb390) [ CSSD]2013-05-23 17:22:00.881 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb390 (RPC#391) tag(187002d) sent to node 1 [ CSSD]2013-05-23 17:22:00.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:00.882 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 391 (0x7f431bfcb390) [ CSSD]2013-05-23 17:22:00.882 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb390 (RPC#391) state 6, flags 0x202 [ CSSD]2013-05-23 17:22:00.882 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcb390, ret 0, client (nil) [ CSSD]2013-05-23 17:22:00.882 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 391 [ CSSD]2013-05-23 17:22:01.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:01.721 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:02.873 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:03.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:03.727 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:03.903 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:03.904 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:04.925 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:05.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:05.738 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:06.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:06.360 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015186, gin 1, member 0 [ CSSD]2013-05-23 17:22:06.360 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:22:06.360 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:22:06.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:06.367 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:22:06.367 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c095670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:22:06.950 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:07.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:07.745 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:07.906 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:07.906 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:22:08.780 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:22:08.782 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:22:08.782 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:22:08.782 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c12b920) pid(22744) version 11:1:1:4 [ CSSD]2013-05-23 17:22:08.788 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:22:08.788 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:22:08.788 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:22:08.788 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:08.788 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 392 (0x7f431bfcb410) [ CSSD]2013-05-23 17:22:08.788 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb410 (RPC#392) tag(188002d) sent to node 1 [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0188002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015187, gin 1, member 0 [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1088c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb410 (RPC#392) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb410, ret 0, client 0x7f431c109f80 member 0x7f431c1216c0 [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:22:08.789 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 392 [ CSSD]2013-05-23 17:22:08.795 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:22:08.795 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:08.795 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1216c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:08.795 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 393 (0x7f431bfcb490) [ CSSD]2013-05-23 17:22:08.795 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb490 (RPC#393) tag(189002d) sent to node 1 [ CSSD]2013-05-23 17:22:08.795 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0189002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:08.795 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:08.795 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb490 (RPC#393) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:08.795 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb490, ret 0, client 0x7f431c109f80 member 0x7f431c1216c0 [ CSSD]2013-05-23 17:22:08.795 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 393 [ CSSD]2013-05-23 17:22:08.795 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1216c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:08.961 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:08.998 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:22:08.998 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c12f0c0) skgpid ospid 22744 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:09.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:09.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:10.022 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 17:22:10.023 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:22:10.024 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 17:22:10.024 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c12b920) pid(22271/22271) version 11:1:1:4 [ CSSD]2013-05-23 17:22:10.030 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:22:10.030 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:22:10.030 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:22:10.030 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 17:22:10.648 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13f950), client(2/0x7f431c0cf720) [ CSSD]2013-05-23 17:22:10.649 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 17:22:10.649 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:22:10.649 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 17:22:10.911 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:22:10.911 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 394 (0x7f431bfcb510) [ CSSD]2013-05-23 17:22:10.912 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb510 (RPC#394) tag(18a002d) sent to node 1 [ CSSD]2013-05-23 17:22:10.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:10.912 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 394 (0x7f431bfcb510) [ CSSD]2013-05-23 17:22:10.912 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb510 (RPC#394) state 6, flags 0x202 [ CSSD]2013-05-23 17:22:10.912 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcb510, ret 0, client (nil) [ CSSD]2013-05-23 17:22:10.912 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 394 [ CSSD]2013-05-23 17:22:10.965 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c12b920), client(2/0x7f431c134bc0) [ CSSD]2013-05-23 17:22:10.965 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c134bc0) [ CSSD]2013-05-23 17:22:10.965 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:22:10.965 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:22:10.967 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:11.740 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f70) allocated [ CSSD]2013-05-23 17:22:11.741 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:22:11.742 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f70 - 1,2,3 [ CSSD]2013-05-23 17:22:11.742 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a70) proc(0x7f431c094f70) pid(22812) version 11:1:1:4 [ CSSD]2013-05-23 17:22:11.758 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c094f70), client(1/0x7f431c134b20) [ CSSD]2013-05-23 17:22:11.758 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134b20) [ CSSD]2013-05-23 17:22:11.758 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134b20 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:22:11.758 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:11.758 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 395 (0x7f431bfcb590) [ CSSD]2013-05-23 17:22:11.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb590 (RPC#395) tag(18b002d) sent to node 1 [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(018b002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015188, gin 1, member 0 [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133070/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb590 (RPC#395) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb590, ret 0, client 0x7f431c134b20 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134b20 [ CSSD]2013-05-23 17:22:11.759 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 395 [ CSSD]2013-05-23 17:22:11.766 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134b20) [ CSSD]2013-05-23 17:22:11.766 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134b20), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:11.766 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:11.766 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 396 (0x7f431bfcb610) [ CSSD]2013-05-23 17:22:11.766 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb610 (RPC#396) tag(18c002d) sent to node 1 [ CSSD]2013-05-23 17:22:11.766 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(018c002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:11.766 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:11.766 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb610 (RPC#396) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:11.766 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb610, ret 0, client 0x7f431c134b20 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:11.767 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 396 [ CSSD]2013-05-23 17:22:11.767 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:11.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:11.821 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:11.969 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f70 [ CSSD]2013-05-23 17:22:11.969 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f70) con(0x7f431c134a70) skgpid ospid 22812 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:12.909 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:12.909 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:13.020 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:13.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:13.846 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:15.026 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:15.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:15.848 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:17.031 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:17.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:17.855 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:17.912 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:17.912 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:19.042 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:19.571 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f70) allocated [ CSSD]2013-05-23 17:22:19.572 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:22:19.572 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f70 - 1,2,3 [ CSSD]2013-05-23 17:22:19.572 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a80) proc(0x7f431c094f70) pid(23060) version 11:1:1:4 [ CSSD]2013-05-23 17:22:19.699 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091640) allocated [ CSSD]2013-05-23 17:22:19.700 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:22:19.701 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091640 - 1,2,3 [ CSSD]2013-05-23 17:22:19.701 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0951e0) proc(0x7f431c091640) pid(23075) version 11:1:1:4 [ CSSD]2013-05-23 17:22:19.720 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:22:19.722 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:22:19.725 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091640 [ CSSD]2013-05-23 17:22:19.725 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091640) con(0x7f431c0951e0) skgpid ospid 23075 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:19.803 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(79/0x7f431c134e00) [ CSSD]2013-05-23 17:22:19.803 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 79 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:19.803 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c134e00 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:22:19.803 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:22:19.803 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 397 (0x7f431bfcb690) [ CSSD]2013-05-23 17:22:19.803 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb690 (RPC#397) tag(18d002d) sent to node 1 [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(018d002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015189, gin 1, member 0 [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c133070/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb690 (RPC#397) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb690, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:22:19.804 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 397 [ CSSD]2013-05-23 17:22:19.811 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 79 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:19.811 [1191196992] >TRACE: clssgmExitGrock: client 79 (0x7f431c134e00), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:22:19.811 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:22:19.811 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 398 (0x7f431bfcb710) [ CSSD]2013-05-23 17:22:19.811 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb710 (RPC#398) tag(18e002d) sent to node 1 [ CSSD]2013-05-23 17:22:19.811 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(018e002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:19.811 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:22:19.811 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb710 (RPC#398) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:19.811 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb710, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:19.811 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 398 [ CSSD]2013-05-23 17:22:19.811 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:19.812 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f70 [ CSSD]2013-05-23 17:22:19.812 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f70) con(0x7f431c134a80) skgpid ospid 23060 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:19.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:19.925 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:21.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:21.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:21.614 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015190, gin 1, member 0 [ CSSD]2013-05-23 17:22:21.615 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c129640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:22:21.615 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:22:21.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:21.622 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:22:21.622 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1349b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:22:21.951 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:21.951 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:22.862 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:22:22.862 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 399 (0x7f431bfcb790) [ CSSD]2013-05-23 17:22:22.862 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb790 (RPC#399) tag(18f002d) sent to node 1 [ CSSD]2013-05-23 17:22:22.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:22.863 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 399 (0x7f431bfcb790) [ CSSD]2013-05-23 17:22:22.863 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb790 (RPC#399) state 6, flags 0x202 [ CSSD]2013-05-23 17:22:22.863 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcb790, ret 0, client (nil) [ CSSD]2013-05-23 17:22:22.863 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 399 [ CSSD]2013-05-23 17:22:22.916 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:22.916 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:23.121 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:23.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:23.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:25.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:25.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:25.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:26.918 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:26.918 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:22:27.004 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f70) allocated [ CSSD]2013-05-23 17:22:27.005 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:22:27.005 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f70 - 1,2,3 [ CSSD]2013-05-23 17:22:27.005 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a80) proc(0x7f431c094f70) pid(23136) version 11:1:1:4 [ CSSD]2013-05-23 17:22:27.011 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c094f70), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:22:27.011 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:27.011 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134e00 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:22:27.011 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:27.011 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 400 (0x7f431bfcb810) [ CSSD]2013-05-23 17:22:27.011 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb810 (RPC#400) tag(190002d) sent to node 1 [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0190002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015191, gin 1, member 0 [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133070/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb810 (RPC#400) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb810, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:22:27.012 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 400 [ CSSD]2013-05-23 17:22:27.018 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:27.018 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134e00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:27.018 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:27.018 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 401 (0x7f431bfcb890) [ CSSD]2013-05-23 17:22:27.018 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb890 (RPC#401) tag(191002d) sent to node 1 [ CSSD]2013-05-23 17:22:27.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0191002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:27.019 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:27.019 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb890 (RPC#401) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:27.019 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcb890, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:27.019 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 401 [ CSSD]2013-05-23 17:22:27.019 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:27.172 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:27.221 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f70 [ CSSD]2013-05-23 17:22:27.221 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f70) con(0x7f431c134a80) skgpid ospid 23136 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:27.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:27.975 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:29.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:30.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:30.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:31.229 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:31.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:31.480 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015192, gin 1, member 0 [ CSSD]2013-05-23 17:22:31.480 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c129640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:22:31.480 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:22:31.488 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:31.488 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:22:31.488 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1349b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:22:31.921 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:31.921 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:32.033 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:32.033 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:32.887 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:22:32.888 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 402 (0x7f431bfcb910) [ CSSD]2013-05-23 17:22:32.888 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb910 (RPC#402) tag(192002d) sent to node 1 [ CSSD]2013-05-23 17:22:32.888 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:32.888 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 402 (0x7f431bfcb910) [ CSSD]2013-05-23 17:22:32.888 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb910 (RPC#402) state 6, flags 0x202 [ CSSD]2013-05-23 17:22:32.888 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcb910, ret 0, client (nil) [ CSSD]2013-05-23 17:22:32.888 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 402 [ CSSD]2013-05-23 17:22:33.240 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:34.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:34.040 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:35.264 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:35.353 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f70) allocated [ CSSD]2013-05-23 17:22:35.354 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:22:35.354 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f70 - 1,2,3 [ CSSD]2013-05-23 17:22:35.355 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a80) proc(0x7f431c094f70) pid(23232) version 11:1:1:4 [ CSSD]2013-05-23 17:22:35.361 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c094f70), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:22:35.361 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:35.361 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134e00 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:22:35.361 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:35.361 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 403 (0x7f431bfcb990) [ CSSD]2013-05-23 17:22:35.361 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcb990 (RPC#403) tag(193002d) sent to node 1 [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0193002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015193, gin 1, member 0 [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133070/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcb990 (RPC#403) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:35.361 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcb990, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:35.362 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:22:35.362 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 403 [ CSSD]2013-05-23 17:22:35.368 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:35.368 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134e00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:35.368 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:35.368 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 404 (0x7f431bfcba10) [ CSSD]2013-05-23 17:22:35.368 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcba10 (RPC#404) tag(194002d) sent to node 1 [ CSSD]2013-05-23 17:22:35.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0194002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:35.368 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:35.368 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcba10 (RPC#404) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:35.368 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcba10, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:35.368 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 404 [ CSSD]2013-05-23 17:22:35.368 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:35.571 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f70 [ CSSD]2013-05-23 17:22:35.571 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f70) con(0x7f431c134a80) skgpid ospid 23232 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:36.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:36.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:36.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:36.875 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015194, gin 1, member 0 [ CSSD]2013-05-23 17:22:36.875 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c129640/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:22:36.875 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:22:36.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:36.883 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:22:36.883 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c134d30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:22:36.924 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:36.924 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:37.269 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:38.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:38.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:39.324 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:40.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:40.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:41.330 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:41.927 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:41.927 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:42.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:42.134 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:42.253 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094ea0) allocated [ CSSD]2013-05-23 17:22:42.254 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:22:42.254 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094ea0 - 1,2,3 [ CSSD]2013-05-23 17:22:42.254 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094ea0) pid(23284) version 11:1:1:4 [ CSSD]2013-05-23 17:22:42.260 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c094ea0), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:22:42.260 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:42.260 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134e00 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:22:42.260 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:42.260 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 405 (0x7f431bfcba90) [ CSSD]2013-05-23 17:22:42.260 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcba90 (RPC#405) tag(195002d) sent to node 1 [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0195002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015195, gin 1, member 0 [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcba90 (RPC#405) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcba90, ret 0, client 0x7f431c134e00 member 0x7f431c094d80 [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:22:42.261 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 405 [ CSSD]2013-05-23 17:22:42.266 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:22:42.266 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134e00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:42.266 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:42.266 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 406 (0x7f431bfcbb10) [ CSSD]2013-05-23 17:22:42.266 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbb10 (RPC#406) tag(196002d) sent to node 1 [ CSSD]2013-05-23 17:22:42.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0196002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:42.267 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:42.267 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbb10 (RPC#406) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:42.267 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcbb10, ret 0, client 0x7f431c134e00 member 0x7f431c094d80 [ CSSD]2013-05-23 17:22:42.267 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 406 [ CSSD]2013-05-23 17:22:42.267 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:42.469 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094ea0 [ CSSD]2013-05-23 17:22:42.469 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094ea0) con(0x7f431c1349b0) skgpid ospid 23284 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:42.927 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:22:42.927 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 407 (0x7f431bfcbb90) [ CSSD]2013-05-23 17:22:42.927 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbb90 (RPC#407) tag(197002d) sent to node 1 [ CSSD]2013-05-23 17:22:42.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:42.928 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 407 (0x7f431bfcbb90) [ CSSD]2013-05-23 17:22:42.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbb90 (RPC#407) state 6, flags 0x202 [ CSSD]2013-05-23 17:22:42.928 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcbb90, ret 0, client (nil) [ CSSD]2013-05-23 17:22:42.928 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 407 [ CSSD]2013-05-23 17:22:43.342 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:44.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:44.140 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:45.347 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:46.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:46.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:46.930 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:46.930 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:47.372 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:48.176 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:48.176 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:49.444 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:50.139 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f8860) [ CSSD]2013-05-23 17:22:50.139 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f8860), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:22:50.139 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f8860), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:22:50.139 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 17:22:50.140 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:22:50.140 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x230f280) skgpid 17956 ospid 17956 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:50.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:50.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:50.933 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:50.933 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:22:51.471 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:52.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:52.137 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015196, gin 1, member 0 [ CSSD]2013-05-23 17:22:52.137 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:22:52.137 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:22:52.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:52.144 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:22:52.144 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:22:52.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:52.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:52.954 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:22:52.954 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 408 (0x7f431bfcbc10) [ CSSD]2013-05-23 17:22:52.954 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbc10 (RPC#408) tag(198002d) sent to node 1 [ CSSD]2013-05-23 17:22:52.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:52.955 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 408 (0x7f431bfcbc10) [ CSSD]2013-05-23 17:22:52.955 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbc10 (RPC#408) state 6, flags 0x202 [ CSSD]2013-05-23 17:22:52.955 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcbc10, ret 0, client (nil) [ CSSD]2013-05-23 17:22:52.955 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 408 [ CSSD]2013-05-23 17:22:53.472 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:54.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:54.240 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:55.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:55.935 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:22:55.936 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:22:56.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:56.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:57.501 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094ea0) allocated [ CSSD]2013-05-23 17:22:57.502 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:22:57.502 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094ea0 - 1,2,3 [ CSSD]2013-05-23 17:22:57.502 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094ea0) pid(23514) version 11:1:1:4 [ CSSD]2013-05-23 17:22:57.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:22:57.508 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c094ea0), client(1/0x7f431c132e60) [ CSSD]2013-05-23 17:22:57.508 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c132e60) [ CSSD]2013-05-23 17:22:57.508 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c132e60 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:22:57.508 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:57.508 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 409 (0x7f431bfcbc90) [ CSSD]2013-05-23 17:22:57.508 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbc90 (RPC#409) tag(199002d) sent to node 1 [ CSSD]2013-05-23 17:22:57.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0199002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:57.508 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015197, gin 1, member 0 [ CSSD]2013-05-23 17:22:57.508 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132fd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:22:57.509 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:22:57.509 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:22:57.509 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbc90 (RPC#409) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:57.509 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcbc90, ret 0, client 0x7f431c132e60 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:57.509 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c132e60 [ CSSD]2013-05-23 17:22:57.509 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 409 [ CSSD]2013-05-23 17:22:57.515 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c132e60) [ CSSD]2013-05-23 17:22:57.515 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c132e60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:57.515 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:22:57.515 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 410 (0x7f431bfcbd10) [ CSSD]2013-05-23 17:22:57.515 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbd10 (RPC#410) tag(19a002d) sent to node 1 [ CSSD]2013-05-23 17:22:57.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(019a002d) incarnation(174631051) [ CSSD]2013-05-23 17:22:57.515 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:22:57.515 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbd10 (RPC#410) state 6, flags 0x100 [ CSSD]2013-05-23 17:22:57.515 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcbd10, ret 0, client 0x7f431c132e60 member 0x7f431c094d60 [ CSSD]2013-05-23 17:22:57.515 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 410 [ CSSD]2013-05-23 17:22:57.515 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:22:57.718 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094ea0 [ CSSD]2013-05-23 17:22:57.718 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094ea0) con(0x7f431c1349b0) skgpid ospid 23514 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:22:58.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:22:58.276 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:22:59.534 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:00.328 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:00.328 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:00.938 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:00.938 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:01.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:02.375 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:02.375 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:02.980 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:23:02.980 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 411 (0x7f431bfcbd90) [ CSSD]2013-05-23 17:23:02.980 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbd90 (RPC#411) tag(19b002d) sent to node 1 [ CSSD]2013-05-23 17:23:02.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:02.981 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 411 (0x7f431bfcbd90) [ CSSD]2013-05-23 17:23:02.981 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbd90 (RPC#411) state 6, flags 0x202 [ CSSD]2013-05-23 17:23:02.981 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcbd90, ret 0, client (nil) [ CSSD]2013-05-23 17:23:02.981 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 411 [ CSSD]2013-05-23 17:23:03.551 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:04.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:04.401 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:05.576 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:05.941 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:05.941 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:06.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:06.431 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:07.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:07.389 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015198, gin 1, member 0 [ CSSD]2013-05-23 17:23:07.389 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:23:07.389 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:23:07.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:07.396 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:23:07.396 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:23:07.582 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:08.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:08.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:09.635 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:10.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:10.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:10.944 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:10.944 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:11.640 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:12.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:12.473 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:12.749 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094ea0) allocated [ CSSD]2013-05-23 17:23:12.750 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:23:12.751 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094ea0 - 1,2,3 [ CSSD]2013-05-23 17:23:12.751 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094ea0) pid(23666) version 11:1:1:4 [ CSSD]2013-05-23 17:23:12.756 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c094ea0), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:23:12.756 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:23:12.756 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134e00 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:23:12.756 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:12.756 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 412 (0x7f431bfcbe10) [ CSSD]2013-05-23 17:23:12.756 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbe10 (RPC#412) tag(19c002d) sent to node 1 [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(019c002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015199, gin 1, member 0 [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbe10 (RPC#412) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcbe10, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:23:12.757 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 412 [ CSSD]2013-05-23 17:23:12.763 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:23:12.763 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134e00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:12.763 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:12.763 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 413 (0x7f431bfcbe90) [ CSSD]2013-05-23 17:23:12.763 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbe90 (RPC#413) tag(19d002d) sent to node 1 [ CSSD]2013-05-23 17:23:12.764 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(019d002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:12.764 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:12.764 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbe90 (RPC#413) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:12.764 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcbe90, ret 0, client 0x7f431c134e00 member 0x7f431c094d60 [ CSSD]2013-05-23 17:23:12.764 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 413 [ CSSD]2013-05-23 17:23:12.764 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:23:12.966 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094ea0 [ CSSD]2013-05-23 17:23:12.966 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094ea0) con(0x7f431c1349b0) skgpid ospid 23666 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:13.003 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:23:13.003 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 414 (0x7f431bfcbf10) [ CSSD]2013-05-23 17:23:13.003 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbf10 (RPC#414) tag(19e002d) sent to node 1 [ CSSD]2013-05-23 17:23:13.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:13.004 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 414 (0x7f431bfcbf10) [ CSSD]2013-05-23 17:23:13.004 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbf10 (RPC#414) state 6, flags 0x202 [ CSSD]2013-05-23 17:23:13.004 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcbf10, ret 0, client (nil) [ CSSD]2013-05-23 17:23:13.004 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 414 [ CSSD]2013-05-23 17:23:13.652 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:14.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:14.526 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:15.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:15.947 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:15.947 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:16.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:16.533 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:17.683 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:18.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:18.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:19.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:20.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:20.549 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:20.950 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:20.950 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:21.759 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:22.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:22.575 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:22.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:22.646 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015200, gin 1, member 0 [ CSSD]2013-05-23 17:23:22.646 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:23:22.646 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:23:22.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:22.654 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:23:22.654 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:23:23.027 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:23:23.028 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 415 (0x7f431bfcbf90) [ CSSD]2013-05-23 17:23:23.028 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcbf90 (RPC#415) tag(19f002d) sent to node 1 [ CSSD]2013-05-23 17:23:23.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:23.028 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 415 (0x7f431bfcbf90) [ CSSD]2013-05-23 17:23:23.028 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcbf90 (RPC#415) state 6, flags 0x202 [ CSSD]2013-05-23 17:23:23.028 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcbf90, ret 0, client (nil) [ CSSD]2013-05-23 17:23:23.028 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 415 [ CSSD]2013-05-23 17:23:23.772 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:24.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:24.575 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:25.797 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:25.953 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:25.953 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:26.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:26.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:27.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:27.998 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f70) allocated [ CSSD]2013-05-23 17:23:28.000 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:23:28.000 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f70 - 1,2,3 [ CSSD]2013-05-23 17:23:28.000 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135890) proc(0x7f431c094f70) pid(23816) version 11:1:1:4 [ CSSD]2013-05-23 17:23:28.006 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c094f70), client(1/0x7f431c095430) [ CSSD]2013-05-23 17:23:28.006 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095430) [ CSSD]2013-05-23 17:23:28.006 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095430 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:23:28.006 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:28.006 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 416 (0x7f431bfcc010) [ CSSD]2013-05-23 17:23:28.006 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc010 (RPC#416) tag(1a0002d) sent to node 1 [ CSSD]2013-05-23 17:23:28.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01a0002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:28.006 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015201, gin 1, member 0 [ CSSD]2013-05-23 17:23:28.006 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133140/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:23:28.007 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:23:28.007 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:23:28.007 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc010 (RPC#416) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:28.007 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc010, ret 0, client 0x7f431c095430 member 0x7f431c094e30 [ CSSD]2013-05-23 17:23:28.007 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095430 [ CSSD]2013-05-23 17:23:28.007 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 416 [ CSSD]2013-05-23 17:23:28.013 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095430) [ CSSD]2013-05-23 17:23:28.013 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095430), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:28.013 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094e30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:28.013 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 417 (0x7f431bfcc090) [ CSSD]2013-05-23 17:23:28.013 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc090 (RPC#417) tag(1a1002d) sent to node 1 [ CSSD]2013-05-23 17:23:28.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01a1002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:28.013 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:28.013 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc090 (RPC#417) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:28.013 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcc090, ret 0, client 0x7f431c095430 member 0x7f431c094e30 [ CSSD]2013-05-23 17:23:28.013 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 417 [ CSSD]2013-05-23 17:23:28.013 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094e30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:23:28.216 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f70 [ CSSD]2013-05-23 17:23:28.216 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f70) con(0x7f431c135890) skgpid ospid 23816 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:28.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:28.608 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:29.835 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:30.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:30.640 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:30.956 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:30.956 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:31.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:31.738 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015202, gin 1, member 0 [ CSSD]2013-05-23 17:23:31.738 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:23:31.738 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:23:31.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:31.744 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:23:31.744 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:23:31.841 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:32.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:32.646 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:33.049 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:23:33.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 418 (0x7f431bfcc110) [ CSSD]2013-05-23 17:23:33.049 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc110 (RPC#418) tag(1a2002d) sent to node 1 [ CSSD]2013-05-23 17:23:33.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:33.050 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 418 (0x7f431bfcc110) [ CSSD]2013-05-23 17:23:33.050 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc110 (RPC#418) state 6, flags 0x202 [ CSSD]2013-05-23 17:23:33.050 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc110, ret 0, client (nil) [ CSSD]2013-05-23 17:23:33.050 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 418 [ CSSD]2013-05-23 17:23:33.853 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:34.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:34.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:34.958 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:34.958 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:23:35.607 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094ea0) allocated [ CSSD]2013-05-23 17:23:35.608 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:23:35.609 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094ea0 - 1,2,3 [ CSSD]2013-05-23 17:23:35.609 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094ea0) pid(23912) version 11:1:1:4 [ CSSD]2013-05-23 17:23:35.614 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c094ea0), client(1/0x7f431c0953f0) [ CSSD]2013-05-23 17:23:35.614 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0953f0) [ CSSD]2013-05-23 17:23:35.614 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0953f0 with con 0x7f431c134df0, requested num -1 [ CSSD]2013-05-23 17:23:35.614 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:35.614 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 419 (0x7f431bfcc190) [ CSSD]2013-05-23 17:23:35.614 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc190 (RPC#419) tag(1a3002d) sent to node 1 [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01a3002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015203, gin 1, member 0 [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc190 (RPC#419) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc190, ret 0, client 0x7f431c0953f0 member 0x7f431c094d60 [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0953f0 [ CSSD]2013-05-23 17:23:35.615 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 419 [ CSSD]2013-05-23 17:23:35.621 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0953f0) [ CSSD]2013-05-23 17:23:35.621 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0953f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:35.621 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:35.621 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 420 (0x7f431bfcc210) [ CSSD]2013-05-23 17:23:35.621 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc210 (RPC#420) tag(1a4002d) sent to node 1 [ CSSD]2013-05-23 17:23:35.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01a4002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:35.621 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:35.621 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc210 (RPC#420) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:35.621 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcc210, ret 0, client 0x7f431c0953f0 member 0x7f431c094d60 [ CSSD]2013-05-23 17:23:35.622 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 420 [ CSSD]2013-05-23 17:23:35.622 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:23:35.824 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094ea0 [ CSSD]2013-05-23 17:23:35.824 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094ea0) con(0x7f431c1349b0) skgpid ospid 23912 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:35.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:36.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:36.694 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:37.885 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:37.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:37.898 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015204, gin 1, member 0 [ CSSD]2013-05-23 17:23:37.898 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:23:37.898 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:23:37.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:37.905 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:23:37.905 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:23:38.720 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:38.720 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:39.937 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:39.961 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:39.961 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:40.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:40.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:41.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:42.259 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:42.259 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015205, gin 1, member 0 [ CSSD]2013-05-23 17:23:42.259 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:23:42.259 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:23:42.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:42.267 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:23:42.267 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:23:42.399 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:23:42.400 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:23:42.401 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:23:42.401 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:23:42.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:23:42.403 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:23:42.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:42.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:43.075 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:23:43.075 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 421 (0x7f431bfcc290) [ CSSD]2013-05-23 17:23:43.075 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc290 (RPC#421) tag(1a5002d) sent to node 1 [ CSSD]2013-05-23 17:23:43.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:43.076 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 421 (0x7f431bfcc290) [ CSSD]2013-05-23 17:23:43.076 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc290 (RPC#421) state 6, flags 0x202 [ CSSD]2013-05-23 17:23:43.076 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc290, ret 0, client (nil) [ CSSD]2013-05-23 17:23:43.076 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 421 [ CSSD]2013-05-23 17:23:43.256 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094ea0) allocated [ CSSD]2013-05-23 17:23:43.257 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:23:43.257 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094ea0 - 1,2,3 [ CSSD]2013-05-23 17:23:43.257 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094ea0) pid(23988) version 11:1:1:4 [ CSSD]2013-05-23 17:23:43.273 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c094ea0), client(1/0x7f431c132e60) [ CSSD]2013-05-23 17:23:43.273 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c132e60) [ CSSD]2013-05-23 17:23:43.273 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c132e60 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:23:43.273 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:43.273 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 422 (0x7f431bfcc310) [ CSSD]2013-05-23 17:23:43.274 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc310 (RPC#422) tag(1a6002d) sent to node 1 [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01a6002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015206, gin 1, member 0 [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132fd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc310 (RPC#422) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc310, ret 0, client 0x7f431c132e60 member 0x7f431c094d60 [ CSSD]2013-05-23 17:23:43.274 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c132e60 [ CSSD]2013-05-23 17:23:43.275 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 422 [ CSSD]2013-05-23 17:23:43.281 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c132e60) [ CSSD]2013-05-23 17:23:43.281 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c132e60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:43.281 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:43.281 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 423 (0x7f431bfcc390) [ CSSD]2013-05-23 17:23:43.281 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc390 (RPC#423) tag(1a7002d) sent to node 1 [ CSSD]2013-05-23 17:23:43.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01a7002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:43.281 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:43.281 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc390 (RPC#423) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:43.281 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcc390, ret 0, client 0x7f431c132e60 member 0x7f431c094d60 [ CSSD]2013-05-23 17:23:43.281 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 423 [ CSSD]2013-05-23 17:23:43.281 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:23:43.484 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094ea0 [ CSSD]2013-05-23 17:23:43.484 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094ea0) con(0x7f431c1349b0) skgpid ospid 23988 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:43.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:44.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:44.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:44.965 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:44.965 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:45.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:46.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:46.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:48.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:48.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:48.799 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:49.968 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:49.968 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:50.037 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:50.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:50.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:52.042 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:52.857 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:52.857 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:53.101 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:23:53.101 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 424 (0x7f431bfcc410) [ CSSD]2013-05-23 17:23:53.101 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc410 (RPC#424) tag(1a8002d) sent to node 1 [ CSSD]2013-05-23 17:23:53.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:53.102 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 424 (0x7f431bfcc410) [ CSSD]2013-05-23 17:23:53.102 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc410 (RPC#424) state 6, flags 0x202 [ CSSD]2013-05-23 17:23:53.102 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc410, ret 0, client (nil) [ CSSD]2013-05-23 17:23:53.102 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 424 [ CSSD]2013-05-23 17:23:53.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:53.153 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015207, gin 1, member 0 [ CSSD]2013-05-23 17:23:53.154 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:23:53.154 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:23:53.160 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:53.160 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:23:53.160 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:23:54.055 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:54.866 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e4c30) [ CSSD]2013-05-23 17:23:54.866 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e4c30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:23:54.866 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e4c30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:23:54.866 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:23:54.866 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e46f0 [ CSSD]2013-05-23 17:23:54.866 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e46f0) con(0x2317d90) skgpid 18636 ospid 18636 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:54.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:54.867 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:54.872 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:23:54.872 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:23:54.872 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:23:54.872 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:23:54.872 [1191196992] >TRACE: clssgmDeadProc: proc 0x205e170 [ CSSD]2013-05-23 17:23:54.872 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x205e170) con(0x22d7350) skgpid 18638 ospid 18638 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:54.883 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f3ef0) [ CSSD]2013-05-23 17:23:54.883 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f3ef0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:23:54.883 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f3ef0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:23:54.883 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:23:54.883 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3dc0 [ CSSD]2013-05-23 17:23:54.883 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3dc0) con(0x22d71f0) skgpid 13258 ospid 13258 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:54.971 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:54.971 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:23:56.080 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:56.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:56.873 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:58.132 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:23:58.516 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f50) allocated [ CSSD]2013-05-23 17:23:58.518 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:23:58.518 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f50 - 1,2,3 [ CSSD]2013-05-23 17:23:58.518 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a80) proc(0x7f431c094f50) pid(24140) version 11:1:1:4 [ CSSD]2013-05-23 17:23:58.524 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c094f50), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:23:58.524 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:23:58.524 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134e00 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:23:58.524 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:58.524 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 425 (0x7f431bfcc490) [ CSSD]2013-05-23 17:23:58.524 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc490 (RPC#425) tag(1a9002d) sent to node 1 [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01a9002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015208, gin 1, member 0 [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133120/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc490 (RPC#425) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc490, ret 0, client 0x7f431c134e00 member 0x7f431c095420 [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:23:58.525 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 425 [ CSSD]2013-05-23 17:23:58.530 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:23:58.530 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134e00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:58.530 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:23:58.530 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 426 (0x7f431bfcc510) [ CSSD]2013-05-23 17:23:58.530 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc510 (RPC#426) tag(1aa002d) sent to node 1 [ CSSD]2013-05-23 17:23:58.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01aa002d) incarnation(174631051) [ CSSD]2013-05-23 17:23:58.531 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:23:58.531 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc510 (RPC#426) state 6, flags 0x100 [ CSSD]2013-05-23 17:23:58.531 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcc510, ret 0, client 0x7f431c134e00 member 0x7f431c095420 [ CSSD]2013-05-23 17:23:58.531 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 426 [ CSSD]2013-05-23 17:23:58.531 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:23:58.733 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f50 [ CSSD]2013-05-23 17:23:58.733 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f50) con(0x7f431c134a80) skgpid ospid 24140 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:23:58.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:23:58.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:23:59.974 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:23:59.974 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:00.138 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:00.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:00.891 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:02.144 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:02.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:02.897 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:03.125 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:24:03.126 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 427 (0x7f431bfcc590) [ CSSD]2013-05-23 17:24:03.126 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc590 (RPC#427) tag(1ab002d) sent to node 1 [ CSSD]2013-05-23 17:24:03.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:03.126 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 427 (0x7f431bfcc590) [ CSSD]2013-05-23 17:24:03.126 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc590 (RPC#427) state 6, flags 0x202 [ CSSD]2013-05-23 17:24:03.126 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc590, ret 0, client (nil) [ CSSD]2013-05-23 17:24:03.127 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 427 [ CSSD]2013-05-23 17:24:04.155 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:04.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:04.948 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:04.977 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:04.977 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:06.160 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:06.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:06.975 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:08.182 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:08.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:08.406 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015209, gin 1, member 0 [ CSSD]2013-05-23 17:24:08.406 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:24:08.406 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:24:08.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:08.413 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:24:08.413 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:24:08.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:08.981 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:09.980 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:09.980 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:10.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:10.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:10.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:12.194 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:12.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:12.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:13.149 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:24:13.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 428 (0x7f431bfcc610) [ CSSD]2013-05-23 17:24:13.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc610 (RPC#428) tag(1ac002d) sent to node 1 [ CSSD]2013-05-23 17:24:13.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:13.150 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 428 (0x7f431bfcc610) [ CSSD]2013-05-23 17:24:13.150 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc610 (RPC#428) state 6, flags 0x202 [ CSSD]2013-05-23 17:24:13.150 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc610, ret 0, client (nil) [ CSSD]2013-05-23 17:24:13.150 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 428 [ CSSD]2013-05-23 17:24:13.765 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094e80) allocated [ CSSD]2013-05-23 17:24:13.766 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:24:13.767 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094e80 - 1,2,3 [ CSSD]2013-05-23 17:24:13.767 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094e80) pid(24303) version 11:1:1:4 [ CSSD]2013-05-23 17:24:13.783 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c094e80), client(1/0x7f431c095470) [ CSSD]2013-05-23 17:24:13.783 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095470) [ CSSD]2013-05-23 17:24:13.783 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095470 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:24:13.783 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:13.783 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 429 (0x7f431bfcc690) [ CSSD]2013-05-23 17:24:13.783 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc690 (RPC#429) tag(1ad002d) sent to node 1 [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01ad002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015210, gin 1, member 0 [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc690 (RPC#429) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc690, ret 0, client 0x7f431c095470 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095470 [ CSSD]2013-05-23 17:24:13.784 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 429 [ CSSD]2013-05-23 17:24:13.790 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095470) [ CSSD]2013-05-23 17:24:13.790 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095470), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:13.790 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:13.790 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 430 (0x7f431bfcc710) [ CSSD]2013-05-23 17:24:13.790 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc710 (RPC#430) tag(1ae002d) sent to node 1 [ CSSD]2013-05-23 17:24:13.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01ae002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:13.791 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:13.791 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc710 (RPC#430) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:13.791 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcc710, ret 0, client 0x7f431c095470 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:13.791 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 430 [ CSSD]2013-05-23 17:24:13.791 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:24:13.993 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094e80 [ CSSD]2013-05-23 17:24:13.993 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094e80) con(0x7f431c1349b0) skgpid ospid 24303 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:24:14.215 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:14.984 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:14.984 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:15.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:15.049 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:16.221 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:17.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:17.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:18.254 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:19.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:19.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:19.987 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:19.988 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:20.259 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:21.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:21.090 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:22.285 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:23.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:23.097 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:23.173 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:24:23.173 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 431 (0x7f431bfcc790) [ CSSD]2013-05-23 17:24:23.174 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc790 (RPC#431) tag(1af002d) sent to node 1 [ CSSD]2013-05-23 17:24:23.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:23.174 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 431 (0x7f431bfcc790) [ CSSD]2013-05-23 17:24:23.174 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc790 (RPC#431) state 6, flags 0x202 [ CSSD]2013-05-23 17:24:23.174 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc790, ret 0, client (nil) [ CSSD]2013-05-23 17:24:23.174 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 431 [ CSSD]2013-05-23 17:24:23.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:23.663 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015211, gin 1, member 0 [ CSSD]2013-05-23 17:24:23.663 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:24:23.663 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:24:23.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:23.670 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:24:23.670 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:24:24.295 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:24.991 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:24.991 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:25.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:25.149 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:26.301 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:27.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:27.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:28.354 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:29.025 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f50) allocated [ CSSD]2013-05-23 17:24:29.026 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:24:29.027 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f50 - 1,2,3 [ CSSD]2013-05-23 17:24:29.027 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135890) proc(0x7f431c094f50) pid(24448) version 11:1:1:4 [ CSSD]2013-05-23 17:24:29.042 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c094f50), client(1/0x7f431c133050) [ CSSD]2013-05-23 17:24:29.042 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c133050) [ CSSD]2013-05-23 17:24:29.042 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c133050 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:24:29.042 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:29.042 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 432 (0x7f431bfcc810) [ CSSD]2013-05-23 17:24:29.042 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc810 (RPC#432) tag(1b0002d) sent to node 1 [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01b0002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015212, gin 1, member 0 [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1331c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc810 (RPC#432) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc810, ret 0, client 0x7f431c133050 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c133050 [ CSSD]2013-05-23 17:24:29.043 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 432 [ CSSD]2013-05-23 17:24:29.049 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c133050) [ CSSD]2013-05-23 17:24:29.049 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c133050), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:29.049 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:29.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 433 (0x7f431bfcc890) [ CSSD]2013-05-23 17:24:29.049 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc890 (RPC#433) tag(1b1002d) sent to node 1 [ CSSD]2013-05-23 17:24:29.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01b1002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:29.050 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:29.050 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc890 (RPC#433) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:29.050 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcc890, ret 0, client 0x7f431c133050 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:29.050 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 433 [ CSSD]2013-05-23 17:24:29.050 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:24:29.180 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:29.180 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:29.252 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f50 [ CSSD]2013-05-23 17:24:29.252 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f50) con(0x7f431c135890) skgpid ospid 24448 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:24:29.516 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:24:29.516 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:24:29.517 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:24:29.518 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:24:29.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:29.608 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015213, gin 1, member 0 [ CSSD]2013-05-23 17:24:29.609 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x205e170/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:24:29.609 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:24:29.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:29.613 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:24:29.613 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:24:29.994 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:29.995 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:30.359 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:31.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:31.273 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:31.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:31.999 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015214, gin 1, member 0 [ CSSD]2013-05-23 17:24:31.999 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:24:31.999 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:24:32.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:32.007 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:24:32.007 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:24:32.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:33.196 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:24:33.196 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 434 (0x7f431bfcc910) [ CSSD]2013-05-23 17:24:33.197 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc910 (RPC#434) tag(1b2002d) sent to node 1 [ CSSD]2013-05-23 17:24:33.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:33.197 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 434 (0x7f431bfcc910) [ CSSD]2013-05-23 17:24:33.197 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc910 (RPC#434) state 6, flags 0x202 [ CSSD]2013-05-23 17:24:33.197 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcc910, ret 0, client (nil) [ CSSD]2013-05-23 17:24:33.197 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 434 [ CSSD]2013-05-23 17:24:33.298 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:33.298 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:33.997 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:33.997 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:24:34.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:35.350 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:35.350 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:35.864 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094e80) allocated [ CSSD]2013-05-23 17:24:35.865 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:24:35.866 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094e80 - 1,2,3 [ CSSD]2013-05-23 17:24:35.866 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094e80) pid(24534) version 11:1:1:4 [ CSSD]2013-05-23 17:24:35.872 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c094e80), client(1/0x7f431c095470) [ CSSD]2013-05-23 17:24:35.872 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095470) [ CSSD]2013-05-23 17:24:35.872 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095470 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:24:35.872 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:35.872 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 435 (0x7f431bfcc990) [ CSSD]2013-05-23 17:24:35.872 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcc990 (RPC#435) tag(1b3002d) sent to node 1 [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01b3002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015215, gin 1, member 0 [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcc990 (RPC#435) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcc990, ret 0, client 0x7f431c095470 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:35.872 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095470 [ CSSD]2013-05-23 17:24:35.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 435 [ CSSD]2013-05-23 17:24:35.878 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095470) [ CSSD]2013-05-23 17:24:35.878 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095470), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:35.878 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:35.878 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 436 (0x7f431bfcca10) [ CSSD]2013-05-23 17:24:35.878 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcca10 (RPC#436) tag(1b4002d) sent to node 1 [ CSSD]2013-05-23 17:24:35.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01b4002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:35.879 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:35.879 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcca10 (RPC#436) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:35.879 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcca10, ret 0, client 0x7f431c095470 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:35.879 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 436 [ CSSD]2013-05-23 17:24:35.879 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:24:36.081 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094e80 [ CSSD]2013-05-23 17:24:36.081 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094e80) con(0x7f431c1349b0) skgpid ospid 24534 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:24:36.382 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:37.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:37.357 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:38.395 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:38.923 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:38.923 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015216, gin 1, member 0 [ CSSD]2013-05-23 17:24:38.923 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:24:38.923 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:24:38.930 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:38.930 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:24:38.930 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:24:39.000 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:39.000 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:39.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:39.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:40.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:41.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:41.474 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:42.407 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:43.220 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:24:43.220 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 437 (0x7f431bfcca90) [ CSSD]2013-05-23 17:24:43.220 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcca90 (RPC#437) tag(1b5002d) sent to node 1 [ CSSD]2013-05-23 17:24:43.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:43.220 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 437 (0x7f431bfcca90) [ CSSD]2013-05-23 17:24:43.220 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcca90 (RPC#437) state 6, flags 0x202 [ CSSD]2013-05-23 17:24:43.221 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcca90, ret 0, client (nil) [ CSSD]2013-05-23 17:24:43.221 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 437 [ CSSD]2013-05-23 17:24:43.500 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:43.500 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:44.004 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:44.004 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:44.293 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094e80) allocated [ CSSD]2013-05-23 17:24:44.294 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:24:44.294 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094e80 - 1,2,3 [ CSSD]2013-05-23 17:24:44.294 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094e80) pid(24621) version 11:1:1:4 [ CSSD]2013-05-23 17:24:44.300 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c094e80), client(1/0x7f431c095400) [ CSSD]2013-05-23 17:24:44.300 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c095400) [ CSSD]2013-05-23 17:24:44.300 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c095400 with con 0x7f431c1214f0, requested num -1 [ CSSD]2013-05-23 17:24:44.300 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:44.300 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 438 (0x7f431bfccb10) [ CSSD]2013-05-23 17:24:44.300 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccb10 (RPC#438) tag(1b6002d) sent to node 1 [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01b6002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015217, gin 1, member 0 [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccb10 (RPC#438) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfccb10, ret 0, client 0x7f431c095400 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c095400 [ CSSD]2013-05-23 17:24:44.301 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 438 [ CSSD]2013-05-23 17:24:44.307 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c095400) [ CSSD]2013-05-23 17:24:44.307 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c095400), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:44.307 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:44.307 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 439 (0x7f431bfccb90) [ CSSD]2013-05-23 17:24:44.307 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccb90 (RPC#439) tag(1b7002d) sent to node 1 [ CSSD]2013-05-23 17:24:44.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01b7002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:44.307 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:44.307 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccb90 (RPC#439) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:44.307 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfccb90, ret 0, client 0x7f431c095400 member 0x7f431c094d60 [ CSSD]2013-05-23 17:24:44.307 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 439 [ CSSD]2013-05-23 17:24:44.307 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:24:44.457 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:44.510 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094e80 [ CSSD]2013-05-23 17:24:44.510 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094e80) con(0x7f431c1349b0) skgpid ospid 24621 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:24:45.500 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:45.500 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:46.461 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:47.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:47.506 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:48.473 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:49.007 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:49.007 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:49.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:49.558 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:50.499 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:51.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:51.564 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:52.505 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:53.251 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:24:53.252 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 440 (0x7f431bfccc10) [ CSSD]2013-05-23 17:24:53.252 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccc10 (RPC#440) tag(1b8002d) sent to node 1 [ CSSD]2013-05-23 17:24:53.252 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:53.252 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 440 (0x7f431bfccc10) [ CSSD]2013-05-23 17:24:53.252 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccc10 (RPC#440) state 6, flags 0x202 [ CSSD]2013-05-23 17:24:53.252 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfccc10, ret 0, client (nil) [ CSSD]2013-05-23 17:24:53.252 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 440 [ CSSD]2013-05-23 17:24:53.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:53.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:54.010 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:54.010 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:54.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:54.181 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015218, gin 1, member 0 [ CSSD]2013-05-23 17:24:54.181 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:24:54.181 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:24:54.188 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:54.188 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:24:54.188 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:24:54.557 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:55.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:55.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:56.563 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:57.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:57.609 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:58.575 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:24:59.013 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:24:59.013 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:24:59.542 [1191196992] >TRACE: clssgmAllocProc: (0x231d6c0) allocated [ CSSD]2013-05-23 17:24:59.543 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:24:59.543 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231d6c0 - 1,2,3 [ CSSD]2013-05-23 17:24:59.543 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2317510) proc(0x231d6c0) pid(24766) version 11:1:1:4 [ CSSD]2013-05-23 17:24:59.560 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231d6c0), client(1/0x2312940) [ CSSD]2013-05-23 17:24:59.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:24:59.560 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312940 with con 0x22d7420, requested num -1 [ CSSD]2013-05-23 17:24:59.560 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:59.560 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 441 (0x7f431bfccc90) [ CSSD]2013-05-23 17:24:59.560 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccc90 (RPC#441) tag(1b9002d) sent to node 1 [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01b9002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015219, gin 1, member 0 [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccc90 (RPC#441) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfccc90, ret 0, client 0x2312940 member 0x22fdb60 [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312940 [ CSSD]2013-05-23 17:24:59.561 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 441 [ CSSD]2013-05-23 17:24:59.568 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312940) [ CSSD]2013-05-23 17:24:59.568 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:59.568 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:24:59.568 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 442 (0x7f431bfccd10) [ CSSD]2013-05-23 17:24:59.568 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccd10 (RPC#442) tag(1ba002d) sent to node 1 [ CSSD]2013-05-23 17:24:59.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01ba002d) incarnation(174631051) [ CSSD]2013-05-23 17:24:59.568 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:24:59.568 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccd10 (RPC#442) state 6, flags 0x100 [ CSSD]2013-05-23 17:24:59.568 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfccd10, ret 0, client 0x2312940 member 0x22fdb60 [ CSSD]2013-05-23 17:24:59.569 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 442 [ CSSD]2013-05-23 17:24:59.569 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:24:59.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:24:59.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:24:59.771 [1191196992] >TRACE: clssgmDeadProc: proc 0x231d6c0 [ CSSD]2013-05-23 17:24:59.771 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231d6c0) con(0x2317510) skgpid ospid 24766 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:00.581 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:01.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:01.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:02.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:03.278 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:25:03.278 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 443 (0x7f431bfccd90) [ CSSD]2013-05-23 17:25:03.278 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccd90 (RPC#443) tag(1bb002d) sent to node 1 [ CSSD]2013-05-23 17:25:03.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:03.279 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 443 (0x7f431bfccd90) [ CSSD]2013-05-23 17:25:03.279 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccd90 (RPC#443) state 6, flags 0x202 [ CSSD]2013-05-23 17:25:03.279 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfccd90, ret 0, client (nil) [ CSSD]2013-05-23 17:25:03.279 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 443 [ CSSD]2013-05-23 17:25:03.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:03.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:04.017 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:04.017 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:04.657 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:05.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:05.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:06.663 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:07.739 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:07.739 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:08.509 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094e80) allocated [ CSSD]2013-05-23 17:25:08.510 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:25:08.511 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094e80 - 1,2,3 [ CSSD]2013-05-23 17:25:08.511 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1349b0) proc(0x7f431c094e80) pid(24881) version 11:1:1:4 [ CSSD]2013-05-23 17:25:08.676 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:09.020 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:09.020 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:09.100 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094e80 [ CSSD]2013-05-23 17:25:09.100 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094e80) con(0x7f431c1349b0) skgpid ospid 24881 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:09.443 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:09.443 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015220, gin 1, member 0 [ CSSD]2013-05-23 17:25:09.443 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:25:09.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:25:09.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:09.451 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:25:09.451 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:25:09.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:09.769 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:10.681 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:11.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:11.776 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:12.726 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:13.303 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:25:13.304 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 444 (0x7f431bfcce10) [ CSSD]2013-05-23 17:25:13.304 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcce10 (RPC#444) tag(1bc002d) sent to node 1 [ CSSD]2013-05-23 17:25:13.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:13.304 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 444 (0x7f431bfcce10) [ CSSD]2013-05-23 17:25:13.305 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcce10 (RPC#444) state 6, flags 0x202 [ CSSD]2013-05-23 17:25:13.305 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcce10, ret 0, client (nil) [ CSSD]2013-05-23 17:25:13.305 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 444 [ CSSD]2013-05-23 17:25:13.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:13.783 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:14.023 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:14.023 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:14.777 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:14.803 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f50) allocated [ CSSD]2013-05-23 17:25:14.804 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:25:14.804 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f50 - 1,2,3 [ CSSD]2013-05-23 17:25:14.804 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a80) proc(0x7f431c094f50) pid(25037) version 11:1:1:4 [ CSSD]2013-05-23 17:25:14.820 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c094f50), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:25:14.820 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:25:14.820 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c134e00 with con 0x7f431c1360e0, requested num -1 [ CSSD]2013-05-23 17:25:14.820 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:14.820 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 445 (0x7f431bfcce90) [ CSSD]2013-05-23 17:25:14.820 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcce90 (RPC#445) tag(1bd002d) sent to node 1 [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01bd002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015221, gin 1, member 0 [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c132f30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcce90 (RPC#445) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcce90, ret 0, client 0x7f431c134e00 member 0x7f431c094e30 [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c134e00 [ CSSD]2013-05-23 17:25:14.821 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 445 [ CSSD]2013-05-23 17:25:14.827 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:25:14.827 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c134e00), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:14.827 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c094e30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:14.827 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 446 (0x7f431bfccf10) [ CSSD]2013-05-23 17:25:14.827 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccf10 (RPC#446) tag(1be002d) sent to node 1 [ CSSD]2013-05-23 17:25:14.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01be002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:14.828 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:14.828 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccf10 (RPC#446) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:14.828 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfccf10, ret 0, client 0x7f431c134e00 member 0x7f431c094e30 [ CSSD]2013-05-23 17:25:14.828 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 446 [ CSSD]2013-05-23 17:25:14.828 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c094e30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:25:15.030 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f50 [ CSSD]2013-05-23 17:25:15.030 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f50) con(0x7f431c134a80) skgpid ospid 25037 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:15.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:15.791 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:16.784 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:17.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:17.817 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:18.796 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:19.027 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:19.027 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:19.870 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:19.870 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:20.172 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2321180) [ CSSD]2013-05-23 17:25:20.172 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2321180), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:25:20.172 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2321180), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:25:20.172 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:25:20.178 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2317c20) [ CSSD]2013-05-23 17:25:20.178 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2317c20), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:25:20.178 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2317c20), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:25:20.178 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:25:20.178 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed180 [ CSSD]2013-05-23 17:25:20.178 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed180) con(0x22dffa0) skgpid 19765 ospid 19765 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:20.819 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:21.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:21.876 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:22.825 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:23.329 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:25:23.330 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 447 (0x7f431bfccf90) [ CSSD]2013-05-23 17:25:23.330 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfccf90 (RPC#447) tag(1bf002d) sent to node 1 [ CSSD]2013-05-23 17:25:23.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:23.330 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 447 (0x7f431bfccf90) [ CSSD]2013-05-23 17:25:23.330 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfccf90 (RPC#447) state 6, flags 0x202 [ CSSD]2013-05-23 17:25:23.330 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfccf90, ret 0, client (nil) [ CSSD]2013-05-23 17:25:23.331 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 447 [ CSSD]2013-05-23 17:25:23.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:23.881 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:24.029 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:24.029 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:24.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:24.694 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015222, gin 1, member 0 [ CSSD]2013-05-23 17:25:24.694 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:25:24.694 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:25:24.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:24.700 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:25:24.700 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:25:24.827 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:25.892 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:25.892 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:26.034 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c094f50) allocated [ CSSD]2013-05-23 17:25:26.035 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:25:26.035 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c094f50 - 1,2,3 [ CSSD]2013-05-23 17:25:26.035 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134a80) proc(0x7f431c094f50) pid(25120/25120) version 11:1:1:4 [ CSSD]2013-05-23 17:25:26.051 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c094f50), client(1/0x7f431c134e00) [ CSSD]2013-05-23 17:25:26.051 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:25:26.051 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:25:26.051 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:25:26.496 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c094f50), client(2/0x7f431c133050) [ CSSD]2013-05-23 17:25:26.496 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c133050) [ CSSD]2013-05-23 17:25:26.496 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:25:26.496 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:25:26.853 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:27.310 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0920e0) allocated [ CSSD]2013-05-23 17:25:27.311 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:25:27.311 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0920e0 - 1,2,3 [ CSSD]2013-05-23 17:25:27.311 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c133960) proc(0x7f431c0920e0) pid(24114/24114) version 11:1:1:4 [ CSSD]2013-05-23 17:25:27.328 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c0920e0), client(1/0x7f431c092510) [ CSSD]2013-05-23 17:25:27.328 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c092510) [ CSSD]2013-05-23 17:25:27.328 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:25:27.328 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:25:27.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:27.897 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:28.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:29.032 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:29.033 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:29.654 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c0920e0), client(2/0x7f431c1363b0) [ CSSD]2013-05-23 17:25:29.654 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c1363b0) [ CSSD]2013-05-23 17:25:29.654 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:25:29.654 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 17:25:29.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:29.910 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:30.074 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082aa0) allocated [ CSSD]2013-05-23 17:25:30.075 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:25:30.076 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082aa0 - 1,2,3 [ CSSD]2013-05-23 17:25:30.076 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c136900) proc(0x7f431c082aa0) pid(25189) version 11:1:1:4 [ CSSD]2013-05-23 17:25:30.081 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c082aa0), client(1/0x7f431c0830d0) [ CSSD]2013-05-23 17:25:30.081 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0830d0) [ CSSD]2013-05-23 17:25:30.081 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0830d0 with con 0x7f431c082bd0, requested num -1 [ CSSD]2013-05-23 17:25:30.081 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:30.081 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 448 (0x7f431bfcd010) [ CSSD]2013-05-23 17:25:30.082 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd010 (RPC#448) tag(1c0002d) sent to node 1 [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01c0002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015223, gin 1, member 0 [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083240/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd010 (RPC#448) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd010, ret 0, client 0x7f431c0830d0 member 0x7f431c082990 [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0830d0 [ CSSD]2013-05-23 17:25:30.083 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 448 [ CSSD]2013-05-23 17:25:30.089 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0830d0) [ CSSD]2013-05-23 17:25:30.089 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0830d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:30.089 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:30.089 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 449 (0x7f431bfcd090) [ CSSD]2013-05-23 17:25:30.089 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd090 (RPC#449) tag(1c1002d) sent to node 1 [ CSSD]2013-05-23 17:25:30.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01c1002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:30.090 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:30.090 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd090 (RPC#449) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:30.090 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd090, ret 0, client 0x7f431c0830d0 member 0x7f431c082990 [ CSSD]2013-05-23 17:25:30.090 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 449 [ CSSD]2013-05-23 17:25:30.090 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c082990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:25:30.293 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082aa0 [ CSSD]2013-05-23 17:25:30.293 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082aa0) con(0x7f431c136900) skgpid ospid 25189 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:30.893 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:31.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:31.935 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:32.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:32.254 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015224, gin 1, member 0 [ CSSD]2013-05-23 17:25:32.254 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:25:32.254 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:25:32.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:32.261 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:25:32.261 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:25:32.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:33.355 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:25:33.355 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 450 (0x7f431bfcd110) [ CSSD]2013-05-23 17:25:33.355 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd110 (RPC#450) tag(1c2002d) sent to node 1 [ CSSD]2013-05-23 17:25:33.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:33.356 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 450 (0x7f431bfcd110) [ CSSD]2013-05-23 17:25:33.356 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd110 (RPC#450) state 6, flags 0x202 [ CSSD]2013-05-23 17:25:33.356 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcd110, ret 0, client (nil) [ CSSD]2013-05-23 17:25:33.356 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 450 [ CSSD]2013-05-23 17:25:33.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:33.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:34.036 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:34.036 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:34.909 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:35.973 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:35.973 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:36.116 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082aa0) allocated [ CSSD]2013-05-23 17:25:36.117 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:25:36.117 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082aa0 - 1,2,3 [ CSSD]2013-05-23 17:25:36.117 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c136900) proc(0x7f431c082aa0) pid(25271) version 11:1:1:4 [ CSSD]2013-05-23 17:25:36.123 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c082aa0), client(1/0x7f431c0830d0) [ CSSD]2013-05-23 17:25:36.123 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0830d0) [ CSSD]2013-05-23 17:25:36.123 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0830d0 with con 0x7f431c082bd0, requested num -1 [ CSSD]2013-05-23 17:25:36.123 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:36.123 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 451 (0x7f431bfcd190) [ CSSD]2013-05-23 17:25:36.123 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd190 (RPC#451) tag(1c3002d) sent to node 1 [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01c3002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015225, gin 1, member 0 [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083240/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd190 (RPC#451) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd190, ret 0, client 0x7f431c0830d0 member 0x7f431c082990 [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0830d0 [ CSSD]2013-05-23 17:25:36.124 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 451 [ CSSD]2013-05-23 17:25:36.130 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0830d0) [ CSSD]2013-05-23 17:25:36.130 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0830d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:36.130 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:36.130 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 452 (0x7f431bfcd210) [ CSSD]2013-05-23 17:25:36.130 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd210 (RPC#452) tag(1c4002d) sent to node 1 [ CSSD]2013-05-23 17:25:36.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01c4002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:36.130 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:36.130 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd210 (RPC#452) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:36.130 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd210, ret 0, client 0x7f431c0830d0 member 0x7f431c082990 [ CSSD]2013-05-23 17:25:36.131 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 452 [ CSSD]2013-05-23 17:25:36.131 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c082990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:25:36.333 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082aa0 [ CSSD]2013-05-23 17:25:36.333 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082aa0) con(0x7f431c136900) skgpid ospid 25271 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:36.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:37.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:37.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:38.937 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:39.038 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:39.039 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:39.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:39.948 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015226, gin 1, member 0 [ CSSD]2013-05-23 17:25:39.948 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:25:39.948 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:25:39.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:39.955 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:25:39.955 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:25:39.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:39.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:40.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:42.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:42.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:42.965 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:43.382 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:25:43.382 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 453 (0x7f431bfcd290) [ CSSD]2013-05-23 17:25:43.382 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd290 (RPC#453) tag(1c5002d) sent to node 1 [ CSSD]2013-05-23 17:25:43.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:43.383 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 453 (0x7f431bfcd290) [ CSSD]2013-05-23 17:25:43.383 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd290 (RPC#453) state 6, flags 0x202 [ CSSD]2013-05-23 17:25:43.383 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcd290, ret 0, client (nil) [ CSSD]2013-05-23 17:25:43.383 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 453 [ CSSD]2013-05-23 17:25:44.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:44.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:44.041 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:44.041 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:44.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:45.335 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082aa0) allocated [ CSSD]2013-05-23 17:25:45.336 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:25:45.336 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082aa0 - 1,2,3 [ CSSD]2013-05-23 17:25:45.336 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0928d0) proc(0x7f431c082aa0) pid(25342) version 11:1:1:4 [ CSSD]2013-05-23 17:25:45.341 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c082aa0), client(1/0x7f431c137e70) [ CSSD]2013-05-23 17:25:45.341 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 17:25:45.341 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137e70 with con 0x7f431c082d00, requested num -1 [ CSSD]2013-05-23 17:25:45.341 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:45.341 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 454 (0x7f431bfcd310) [ CSSD]2013-05-23 17:25:45.341 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd310 (RPC#454) tag(1c6002d) sent to node 1 [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01c6002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015227, gin 1, member 0 [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0832d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd310 (RPC#454) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd310, ret 0, client 0x7f431c137e70 member 0x7f431c082990 [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137e70 [ CSSD]2013-05-23 17:25:45.342 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 454 [ CSSD]2013-05-23 17:25:45.349 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137e70) [ CSSD]2013-05-23 17:25:45.349 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:45.349 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:25:45.349 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 455 (0x7f431bfcd390) [ CSSD]2013-05-23 17:25:45.349 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd390 (RPC#455) tag(1c7002d) sent to node 1 [ CSSD]2013-05-23 17:25:45.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01c7002d) incarnation(174631051) [ CSSD]2013-05-23 17:25:45.349 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:25:45.349 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd390 (RPC#455) state 6, flags 0x100 [ CSSD]2013-05-23 17:25:45.349 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd390, ret 0, client 0x7f431c137e70 member 0x7f431c082990 [ CSSD]2013-05-23 17:25:45.350 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 455 [ CSSD]2013-05-23 17:25:45.350 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c082990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:25:45.552 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082aa0 [ CSSD]2013-05-23 17:25:45.552 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082aa0) con(0x7f431c0928d0) skgpid ospid 25342 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:25:46.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:46.074 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:47.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:48.080 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:48.080 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:49.017 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:49.044 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:49.044 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:50.091 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:50.091 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:51.023 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:52.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:52.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:53.075 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:53.404 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:25:53.405 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 456 (0x7f431bfcd410) [ CSSD]2013-05-23 17:25:53.405 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd410 (RPC#456) tag(1c8002d) sent to node 1 [ CSSD]2013-05-23 17:25:53.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:53.406 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 456 (0x7f431bfcd410) [ CSSD]2013-05-23 17:25:53.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd410 (RPC#456) state 6, flags 0x202 [ CSSD]2013-05-23 17:25:53.406 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcd410, ret 0, client (nil) [ CSSD]2013-05-23 17:25:53.406 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 456 [ CSSD]2013-05-23 17:25:54.048 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:54.048 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:25:54.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:54.104 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:55.080 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:55.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:55.200 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015228, gin 1, member 0 [ CSSD]2013-05-23 17:25:55.200 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:25:55.200 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:25:55.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:55.207 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:25:55.207 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:25:56.114 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:56.114 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:57.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:25:58.050 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:25:58.050 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:25:58.141 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:25:58.141 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:25:59.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:00.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:00.171 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:00.584 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082a10) allocated [ CSSD]2013-05-23 17:26:00.585 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:26:00.586 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082a10 - 1,2,3 [ CSSD]2013-05-23 17:26:00.586 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1362a0) proc(0x7f431c082a10) pid(25492) version 11:1:1:4 [ CSSD]2013-05-23 17:26:00.592 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c082a10), client(1/0x7f431c083190) [ CSSD]2013-05-23 17:26:00.592 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c083190) [ CSSD]2013-05-23 17:26:00.592 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c083190 with con 0x7f431c082c90, requested num -1 [ CSSD]2013-05-23 17:26:00.592 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:00.592 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 457 (0x7f431bfcd490) [ CSSD]2013-05-23 17:26:00.592 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd490 (RPC#457) tag(1c9002d) sent to node 1 [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01c9002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015229, gin 1, member 0 [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083300/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd490 (RPC#457) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd490, ret 0, client 0x7f431c083190 member 0x7f431c082900 [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c083190 [ CSSD]2013-05-23 17:26:00.593 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 457 [ CSSD]2013-05-23 17:26:00.600 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c083190) [ CSSD]2013-05-23 17:26:00.600 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c083190), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:00.600 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082900) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:00.600 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 458 (0x7f431bfcd510) [ CSSD]2013-05-23 17:26:00.600 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd510 (RPC#458) tag(1ca002d) sent to node 1 [ CSSD]2013-05-23 17:26:00.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01ca002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:00.600 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:00.600 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd510 (RPC#458) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:00.600 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd510, ret 0, client 0x7f431c083190 member 0x7f431c082900 [ CSSD]2013-05-23 17:26:00.600 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 458 [ CSSD]2013-05-23 17:26:00.601 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c082900) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:26:00.803 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082a10 [ CSSD]2013-05-23 17:26:00.803 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082a10) con(0x7f431c1362a0) skgpid ospid 25492 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:01.123 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:02.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:02.177 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:03.053 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:03.053 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:03.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:03.428 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:26:03.429 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 459 (0x7f431bfcd590) [ CSSD]2013-05-23 17:26:03.429 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd590 (RPC#459) tag(1cb002d) sent to node 1 [ CSSD]2013-05-23 17:26:03.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:03.429 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 459 (0x7f431bfcd590) [ CSSD]2013-05-23 17:26:03.429 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd590 (RPC#459) state 6, flags 0x202 [ CSSD]2013-05-23 17:26:03.429 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcd590, ret 0, client (nil) [ CSSD]2013-05-23 17:26:03.430 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 459 [ CSSD]2013-05-23 17:26:04.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:04.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:05.181 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:06.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:06.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:07.187 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:08.056 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:08.057 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:08.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:08.222 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:09.198 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:10.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:10.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:10.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:10.452 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015230, gin 1, member 0 [ CSSD]2013-05-23 17:26:10.452 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:26:10.452 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:26:10.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:10.460 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:26:10.460 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:26:11.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:12.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:12.240 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:13.060 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:13.060 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:13.276 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:13.453 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:26:13.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 460 (0x7f431bfcd610) [ CSSD]2013-05-23 17:26:13.454 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd610 (RPC#460) tag(1cc002d) sent to node 1 [ CSSD]2013-05-23 17:26:13.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:13.454 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 460 (0x7f431bfcd610) [ CSSD]2013-05-23 17:26:13.454 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd610 (RPC#460) state 6, flags 0x202 [ CSSD]2013-05-23 17:26:13.454 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcd610, ret 0, client (nil) [ CSSD]2013-05-23 17:26:13.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 460 [ CSSD]2013-05-23 17:26:14.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:14.267 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:15.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:15.835 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082aa0) allocated [ CSSD]2013-05-23 17:26:15.836 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:26:15.837 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082aa0 - 1,2,3 [ CSSD]2013-05-23 17:26:15.837 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c136900) proc(0x7f431c082aa0) pid(25637) version 11:1:1:4 [ CSSD]2013-05-23 17:26:15.843 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c082aa0), client(1/0x7f431c083160) [ CSSD]2013-05-23 17:26:15.843 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c083160) [ CSSD]2013-05-23 17:26:15.843 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c083160 with con 0x7f431c082c60, requested num -1 [ CSSD]2013-05-23 17:26:15.843 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:15.843 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 461 (0x7f431bfcd690) [ CSSD]2013-05-23 17:26:15.843 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd690 (RPC#461) tag(1cd002d) sent to node 1 [ CSSD]2013-05-23 17:26:15.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01cd002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:15.843 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015231, gin 1, member 0 [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0832d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd690 (RPC#461) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd690, ret 0, client 0x7f431c083160 member 0x7f431c082990 [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c083160 [ CSSD]2013-05-23 17:26:15.844 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 461 [ CSSD]2013-05-23 17:26:15.850 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c083160) [ CSSD]2013-05-23 17:26:15.850 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c083160), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:15.850 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082990) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:15.850 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 462 (0x7f431bfcd710) [ CSSD]2013-05-23 17:26:15.850 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd710 (RPC#462) tag(1ce002d) sent to node 1 [ CSSD]2013-05-23 17:26:15.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01ce002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:15.850 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:15.850 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd710 (RPC#462) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:15.850 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd710, ret 0, client 0x7f431c083160 member 0x7f431c082990 [ CSSD]2013-05-23 17:26:15.851 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 462 [ CSSD]2013-05-23 17:26:15.851 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c082990) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:26:16.053 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082aa0 [ CSSD]2013-05-23 17:26:16.053 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082aa0) con(0x7f431c136900) skgpid ospid 25637 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:16.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:16.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:17.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:18.063 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:18.063 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:18.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:18.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:19.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:20.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:20.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:21.195 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082990) allocated [ CSSD]2013-05-23 17:26:21.197 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:26:21.197 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082990 - 1,2,3 [ CSSD]2013-05-23 17:26:21.197 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1331c0) proc(0x7f431c082990) pid(25732) version 11:1:1:4 [ CSSD]2013-05-23 17:26:21.206 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c082990), client(1/0x7f431c083090) [ CSSD]2013-05-23 17:26:21.206 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c083090) [ CSSD]2013-05-23 17:26:21.206 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c083090 with con 0x7f431c082b90, requested num -1 [ CSSD]2013-05-23 17:26:21.206 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:26:21.206 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 463 (0x7f431bfcd790) [ CSSD]2013-05-23 17:26:21.206 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd790 (RPC#463) tag(1cf002d) sent to node 1 [ CSSD]2013-05-23 17:26:21.206 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01cf002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:21.206 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015232, gin 1, member 0 [ CSSD]2013-05-23 17:26:21.206 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083200/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:26:21.207 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:26:21.207 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:26:21.207 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd790 (RPC#463) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:21.207 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd790, ret 0, client 0x7f431c083090 member 0x7f431c082880 [ CSSD]2013-05-23 17:26:21.207 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c083090 [ CSSD]2013-05-23 17:26:21.207 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 463 [ CSSD]2013-05-23 17:26:21.214 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c083090) [ CSSD]2013-05-23 17:26:21.214 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c083090), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:26:21.214 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c082880) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:26:21.214 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 464 (0x7f431bfcd810) [ CSSD]2013-05-23 17:26:21.214 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd810 (RPC#464) tag(1d0002d) sent to node 1 [ CSSD]2013-05-23 17:26:21.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01d0002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:21.215 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:26:21.215 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd810 (RPC#464) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:21.215 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd810, ret 0, client 0x7f431c083090 member 0x7f431c082880 [ CSSD]2013-05-23 17:26:21.215 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 464 [ CSSD]2013-05-23 17:26:21.215 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c082880) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:26:21.302 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b880) allocated [ CSSD]2013-05-23 17:26:21.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:21.303 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:26:21.303 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b880 - 1,2,3 [ CSSD]2013-05-23 17:26:21.303 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c083120) proc(0x7f431c13b880) pid(25740) version 11:1:1:4 [ CSSD]2013-05-23 17:26:21.328 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:26:21.330 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:26:21.333 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b880 [ CSSD]2013-05-23 17:26:21.333 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b880) con(0x7f431c083120) skgpid ospid 25740 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:21.417 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082990 [ CSSD]2013-05-23 17:26:21.417 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082990) con(0x7f431c1331c0) skgpid ospid 25732 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:22.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:22.360 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:23.066 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:23.066 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:23.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:23.476 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:26:23.476 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 465 (0x7f431bfcd890) [ CSSD]2013-05-23 17:26:23.476 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd890 (RPC#465) tag(1d1002d) sent to node 1 [ CSSD]2013-05-23 17:26:23.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:23.477 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 465 (0x7f431bfcd890) [ CSSD]2013-05-23 17:26:23.477 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd890 (RPC#465) state 6, flags 0x202 [ CSSD]2013-05-23 17:26:23.477 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcd890, ret 0, client (nil) [ CSSD]2013-05-23 17:26:23.477 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 465 [ CSSD]2013-05-23 17:26:24.386 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:24.386 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:24.928 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:24.928 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:26:24.928 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:26:24.928 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 17:26:24.936 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:26:24.936 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:26:24.936 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:26:24.936 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:26:24.936 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:26:24.936 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e6720) skgpid 20478 ospid 20478 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:25.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:25.707 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:25.707 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015233, gin 1, member 0 [ CSSD]2013-05-23 17:26:25.707 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:26:25.707 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:26:25.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:25.714 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:26:25.714 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:26:26.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:26.396 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:27.407 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:28.070 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:28.070 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:28.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:28.403 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:29.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:30.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:30.432 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:31.089 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:26:31.090 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:26:31.091 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:26:31.091 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c10a490) pid(25844) version 11:1:1:4 [ CSSD]2013-05-23 17:26:31.108 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:31.108 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:31.108 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:26:31.108 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:31.108 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 466 (0x7f431bfcd910) [ CSSD]2013-05-23 17:26:31.108 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd910 (RPC#466) tag(1d2002d) sent to node 1 [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01d2002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015234, gin 1, member 0 [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c136520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd910 (RPC#466) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcd910, ret 0, client 0x7f431c12e6d0 member 0x7f431c135290 [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:26:31.109 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 466 [ CSSD]2013-05-23 17:26:31.117 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:31.117 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:31.117 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135290) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:31.117 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 467 (0x7f431bfcd990) [ CSSD]2013-05-23 17:26:31.117 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcd990 (RPC#467) tag(1d3002d) sent to node 1 [ CSSD]2013-05-23 17:26:31.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01d3002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:31.117 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:31.117 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcd990 (RPC#467) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:31.117 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcd990, ret 0, client 0x7f431c12e6d0 member 0x7f431c135290 [ CSSD]2013-05-23 17:26:31.118 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 467 [ CSSD]2013-05-23 17:26:31.118 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135290) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:26:31.320 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:26:31.320 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0840c0) skgpid ospid 25844 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:31.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:32.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:32.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:32.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:32.522 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015235, gin 1, member 0 [ CSSD]2013-05-23 17:26:32.522 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:26:32.522 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:26:32.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:32.530 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:26:32.530 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:26:33.073 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:33.073 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:33.500 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:26:33.500 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 468 (0x7f431bfcda10) [ CSSD]2013-05-23 17:26:33.500 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcda10 (RPC#468) tag(1d4002d) sent to node 1 [ CSSD]2013-05-23 17:26:33.501 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:33.501 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 468 (0x7f431bfcda10) [ CSSD]2013-05-23 17:26:33.501 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcda10 (RPC#468) state 6, flags 0x202 [ CSSD]2013-05-23 17:26:33.501 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcda10, ret 0, client (nil) [ CSSD]2013-05-23 17:26:33.501 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 468 [ CSSD]2013-05-23 17:26:33.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:34.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:34.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:35.523 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:36.380 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:26:36.381 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:26:36.381 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:26:36.381 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c10a490) pid(25919) version 11:1:1:4 [ CSSD]2013-05-23 17:26:36.399 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:36.399 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:36.399 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:26:36.399 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:36.399 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 469 (0x7f431bfcda90) [ CSSD]2013-05-23 17:26:36.399 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcda90 (RPC#469) tag(1d5002d) sent to node 1 [ CSSD]2013-05-23 17:26:36.399 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01d5002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:36.399 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015236, gin 1, member 0 [ CSSD]2013-05-23 17:26:36.399 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c136520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:26:36.400 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:26:36.400 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:26:36.400 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcda90 (RPC#469) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:36.400 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcda90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:26:36.400 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:26:36.400 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 469 [ CSSD]2013-05-23 17:26:36.406 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:36.406 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:36.406 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129c20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:36.406 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 470 (0x7f431bfcdb10) [ CSSD]2013-05-23 17:26:36.406 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdb10 (RPC#470) tag(1d6002d) sent to node 1 [ CSSD]2013-05-23 17:26:36.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01d6002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:36.406 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:36.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdb10 (RPC#470) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:36.406 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcdb10, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:26:36.406 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 470 [ CSSD]2013-05-23 17:26:36.406 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129c20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:26:36.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:36.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:36.609 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:26:36.609 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e6720) skgpid ospid 25919 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:37.076 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:37.076 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:26:37.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:38.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:38.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:39.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:40.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:40.534 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:40.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:40.958 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015237, gin 1, member 0 [ CSSD]2013-05-23 17:26:40.958 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:26:40.958 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:26:40.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:40.965 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:26:40.965 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:26:41.585 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:42.079 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:42.079 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:42.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:42.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:43.525 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:26:43.525 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 471 (0x7f431bfcdb90) [ CSSD]2013-05-23 17:26:43.525 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdb90 (RPC#471) tag(1d7002d) sent to node 1 [ CSSD]2013-05-23 17:26:43.526 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:43.526 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 471 (0x7f431bfcdb90) [ CSSD]2013-05-23 17:26:43.526 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdb90 (RPC#471) state 6, flags 0x202 [ CSSD]2013-05-23 17:26:43.526 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcdb90, ret 0, client (nil) [ CSSD]2013-05-23 17:26:43.526 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 471 [ CSSD]2013-05-23 17:26:43.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:44.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:44.592 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:45.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:46.351 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:26:46.352 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:26:46.352 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:26:46.352 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c10a490) pid(26011) version 11:1:1:4 [ CSSD]2013-05-23 17:26:46.358 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:46.358 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:46.358 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 17:26:46.358 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:46.358 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 472 (0x7f431bfcdc10) [ CSSD]2013-05-23 17:26:46.358 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdc10 (RPC#472) tag(1d8002d) sent to node 1 [ CSSD]2013-05-23 17:26:46.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01d8002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:46.358 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015238, gin 1, member 0 [ CSSD]2013-05-23 17:26:46.358 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c136520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:26:46.358 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:26:46.358 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:26:46.358 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdc10 (RPC#472) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:46.359 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcdc10, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:26:46.359 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:26:46.359 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 472 [ CSSD]2013-05-23 17:26:46.364 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:26:46.365 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:46.365 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129c20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:26:46.365 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 473 (0x7f431bfcdc90) [ CSSD]2013-05-23 17:26:46.365 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdc90 (RPC#473) tag(1d9002d) sent to node 1 [ CSSD]2013-05-23 17:26:46.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01d9002d) incarnation(174631051) [ CSSD]2013-05-23 17:26:46.365 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:26:46.365 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdc90 (RPC#473) state 6, flags 0x100 [ CSSD]2013-05-23 17:26:46.365 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcdc90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:26:46.365 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 473 [ CSSD]2013-05-23 17:26:46.365 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129c20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:26:46.568 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:26:46.568 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e5200) skgpid ospid 26011 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:26:46.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:46.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:47.082 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:47.082 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:47.626 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:48.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:48.623 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:49.629 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:50.633 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:50.633 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:51.654 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:52.085 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:52.085 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:52.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:52.640 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:53.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:26:53.556 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 474 (0x7f431bfcdd10) [ CSSD]2013-05-23 17:26:53.556 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdd10 (RPC#474) tag(1da002d) sent to node 1 [ CSSD]2013-05-23 17:26:53.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:53.557 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 474 (0x7f431bfcdd10) [ CSSD]2013-05-23 17:26:53.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdd10 (RPC#474) state 6, flags 0x202 [ CSSD]2013-05-23 17:26:53.557 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcdd10, ret 0, client (nil) [ CSSD]2013-05-23 17:26:53.557 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 474 [ CSSD]2013-05-23 17:26:53.687 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:54.692 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:54.692 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:55.693 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:56.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:56.219 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015239, gin 1, member 0 [ CSSD]2013-05-23 17:26:56.219 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:26:56.219 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:26:56.226 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:56.226 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:26:56.226 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:26:56.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:56.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:57.088 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:26:57.088 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:26:57.698 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:26:58.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:26:58.724 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:26:59.709 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:00.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:00.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:01.090 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:01.091 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:27:01.599 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:27:01.600 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:27:01.601 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:27:01.601 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c10a490) pid(26158) version 11:1:1:4 [ CSSD]2013-05-23 17:27:01.607 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:01.607 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:01.607 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 17:27:01.607 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:01.607 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 475 (0x7f431bfcdd90) [ CSSD]2013-05-23 17:27:01.607 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdd90 (RPC#475) tag(1db002d) sent to node 1 [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01db002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015240, gin 1, member 0 [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c136520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdd90 (RPC#475) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcdd90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:27:01.608 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 475 [ CSSD]2013-05-23 17:27:01.615 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:01.615 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:01.615 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129c20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:01.615 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 476 (0x7f431bfcde10) [ CSSD]2013-05-23 17:27:01.615 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcde10 (RPC#476) tag(1dc002d) sent to node 1 [ CSSD]2013-05-23 17:27:01.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01dc002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:01.615 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:01.615 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcde10 (RPC#476) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:01.615 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcde10, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:27:01.615 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 476 [ CSSD]2013-05-23 17:27:01.615 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129c20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:27:01.734 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:01.818 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:27:01.818 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e5200) skgpid ospid 26158 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:02.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:02.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:03.586 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:27:03.586 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 477 (0x7f431bfcde90) [ CSSD]2013-05-23 17:27:03.586 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcde90 (RPC#477) tag(1dd002d) sent to node 1 [ CSSD]2013-05-23 17:27:03.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:03.587 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 477 (0x7f431bfcde90) [ CSSD]2013-05-23 17:27:03.587 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcde90 (RPC#477) state 6, flags 0x202 [ CSSD]2013-05-23 17:27:03.587 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcde90, ret 0, client (nil) [ CSSD]2013-05-23 17:27:03.587 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 477 [ CSSD]2013-05-23 17:27:03.806 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:04.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:04.793 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:05.812 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:06.094 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:06.094 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:06.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:06.819 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:07.817 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:08.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:08.825 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:09.830 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:10.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:10.835 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:11.097 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:11.097 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:11.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:11.470 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015241, gin 1, member 0 [ CSSD]2013-05-23 17:27:11.470 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:11.470 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:11.476 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:11.476 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:11.476 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:11.836 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:12.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:12.841 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:13.617 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:27:13.617 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 478 (0x7f431bfcdf10) [ CSSD]2013-05-23 17:27:13.617 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdf10 (RPC#478) tag(1de002d) sent to node 1 [ CSSD]2013-05-23 17:27:13.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:13.618 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 478 (0x7f431bfcdf10) [ CSSD]2013-05-23 17:27:13.618 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdf10 (RPC#478) state 6, flags 0x202 [ CSSD]2013-05-23 17:27:13.618 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcdf10, ret 0, client (nil) [ CSSD]2013-05-23 17:27:13.618 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 478 [ CSSD]2013-05-23 17:27:13.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:14.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:14.295 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015242, gin 1, member 0 [ CSSD]2013-05-23 17:27:14.295 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:14.295 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:14.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:14.304 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:14.304 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:14.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:14.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:15.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:16.100 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:16.100 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:16.850 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:27:16.851 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:27:16.851 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:27:16.851 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c10a490) pid(26518) version 11:1:1:4 [ CSSD]2013-05-23 17:27:16.857 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:16.857 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:16.857 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:27:16.857 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:16.857 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 479 (0x7f431bfcdf90) [ CSSD]2013-05-23 17:27:16.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcdf90 (RPC#479) tag(1df002d) sent to node 1 [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01df002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015243, gin 1, member 0 [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c136520/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcdf90 (RPC#479) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcdf90, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:27:16.858 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:27:16.859 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 479 [ CSSD]2013-05-23 17:27:16.865 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:16.865 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:16.865 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129c20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:16.865 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 480 (0x7f431bfce010) [ CSSD]2013-05-23 17:27:16.865 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce010 (RPC#480) tag(1e0002d) sent to node 1 [ CSSD]2013-05-23 17:27:16.866 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01e0002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:16.866 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:16.866 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce010 (RPC#480) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:16.866 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce010, ret 0, client 0x7f431c12e6d0 member 0x7f431c129c20 [ CSSD]2013-05-23 17:27:16.866 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 480 [ CSSD]2013-05-23 17:27:16.866 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c129c20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:27:16.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:16.906 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:17.068 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:27:17.068 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e5200) skgpid ospid 26518 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:17.919 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:18.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:18.904 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:19.898 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:27:19.899 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:27:19.900 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:27:19.900 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c10a490) pid(26577) version 11:1:1:4 [ CSSD]2013-05-23 17:27:19.930 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:20.032 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:27:20.033 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:27:20.034 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:27:20.034 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c10a190) pid(26587) version 11:1:1:4 [ CSSD]2013-05-23 17:27:20.050 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:27:20.051 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:27:20.053 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:27:20.053 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e6720) skgpid ospid 26587 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:20.102 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:20.102 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:27:20.128 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(80/0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:20.128 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 80 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:20.128 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c12e6d0 with con 0x7f431c0cf470, requested num -1 [ CSSD]2013-05-23 17:27:20.128 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:27:20.128 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 481 (0x7f431bfce090) [ CSSD]2013-05-23 17:27:20.128 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce090 (RPC#481) tag(1e1002d) sent to node 1 [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01e1002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015244, gin 1, member 0 [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c1367e0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce090 (RPC#481) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfce090, ret 0, client 0x7f431c12e6d0 member 0x7f431c0923d0 [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:27:20.129 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 481 [ CSSD]2013-05-23 17:27:20.136 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 80 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:20.136 [1191196992] >TRACE: clssgmExitGrock: client 80 (0x7f431c12e6d0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:27:20.136 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0923d0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:27:20.136 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 482 (0x7f431bfce110) [ CSSD]2013-05-23 17:27:20.136 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce110 (RPC#482) tag(1e2002d) sent to node 1 [ CSSD]2013-05-23 17:27:20.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01e2002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:20.136 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:27:20.136 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce110 (RPC#482) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:20.136 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce110, ret 0, client 0x7f431c12e6d0 member 0x7f431c0923d0 [ CSSD]2013-05-23 17:27:20.137 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 482 [ CSSD]2013-05-23 17:27:20.137 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c0923d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:27:20.138 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:27:20.138 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0e5200) skgpid ospid 26577 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:20.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:20.915 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:21.936 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:22.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:22.941 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:23.647 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:27:23.647 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 483 (0x7f431bfce190) [ CSSD]2013-05-23 17:27:23.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce190 (RPC#483) tag(1e3002d) sent to node 1 [ CSSD]2013-05-23 17:27:23.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:23.648 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 483 (0x7f431bfce190) [ CSSD]2013-05-23 17:27:23.648 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce190 (RPC#483) state 6, flags 0x202 [ CSSD]2013-05-23 17:27:23.648 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce190, ret 0, client (nil) [ CSSD]2013-05-23 17:27:23.648 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 483 [ CSSD]2013-05-23 17:27:23.988 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:24.159 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:27:24.160 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:27:24.160 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:27:24.160 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c10a490) pid(25785/25785) version 11:1:1:4 [ CSSD]2013-05-23 17:27:24.176 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:24.176 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:27:24.176 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:27:24.176 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:27:25.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:25.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:25.105 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:25.106 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:25.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:26.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:26.728 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015245, gin 1, member 0 [ CSSD]2013-05-23 17:27:26.728 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:26.728 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:26.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:26.734 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:26.734 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:27.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:27.020 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:27.822 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0826c0) allocated [ CSSD]2013-05-23 17:27:27.823 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:27:27.823 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0826c0 - 1,2,3 [ CSSD]2013-05-23 17:27:27.823 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c0826c0) pid(25783/25783) version 11:1:1:4 [ CSSD]2013-05-23 17:27:27.839 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c0826c0), client(1/0x7f431c0e5200) [ CSSD]2013-05-23 17:27:27.839 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e5200) [ CSSD]2013-05-23 17:27:27.839 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:27:27.839 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:27:27.999 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:29.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:29.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:30.011 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:30.109 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:30.109 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:31.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:31.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:32.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:32.100 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b950) allocated [ CSSD]2013-05-23 17:27:32.101 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:27:32.101 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b950 - 1,2,3 [ CSSD]2013-05-23 17:27:32.101 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c083250) proc(0x7f431c13b950) pid(26676) version 11:1:1:4 [ CSSD]2013-05-23 17:27:32.108 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13b950), client(1/0x7f431c13beb0) [ CSSD]2013-05-23 17:27:32.108 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13beb0) [ CSSD]2013-05-23 17:27:32.108 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13beb0 with con 0x7f431c13bbb0, requested num -1 [ CSSD]2013-05-23 17:27:32.108 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:32.108 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 484 (0x7f431bfce210) [ CSSD]2013-05-23 17:27:32.108 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce210 (RPC#484) tag(1e4002d) sent to node 1 [ CSSD]2013-05-23 17:27:32.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01e4002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:32.108 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015246, gin 1, member 0 [ CSSD]2013-05-23 17:27:32.108 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:27:32.109 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:27:32.109 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:27:32.109 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce210 (RPC#484) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:32.109 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfce210, ret 0, client 0x7f431c13beb0 member 0x7f431c13b840 [ CSSD]2013-05-23 17:27:32.109 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13beb0 [ CSSD]2013-05-23 17:27:32.109 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 484 [ CSSD]2013-05-23 17:27:32.115 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13beb0) [ CSSD]2013-05-23 17:27:32.115 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13beb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:32.115 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:32.115 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 485 (0x7f431bfce290) [ CSSD]2013-05-23 17:27:32.116 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce290 (RPC#485) tag(1e5002d) sent to node 1 [ CSSD]2013-05-23 17:27:32.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01e5002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:32.116 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:32.116 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce290 (RPC#485) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:32.116 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce290, ret 0, client 0x7f431c13beb0 member 0x7f431c13b840 [ CSSD]2013-05-23 17:27:32.116 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 485 [ CSSD]2013-05-23 17:27:32.116 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:27:32.319 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b950 [ CSSD]2013-05-23 17:27:32.319 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b950) con(0x7f431c083250) skgpid ospid 26676 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:32.782 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:32.782 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015247, gin 1, member 0 [ CSSD]2013-05-23 17:27:32.782 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:32.782 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:32.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:32.787 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:32.787 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:33.042 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:33.042 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:33.666 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:27:33.667 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 486 (0x7f431bfce310) [ CSSD]2013-05-23 17:27:33.667 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce310 (RPC#486) tag(1e6002d) sent to node 1 [ CSSD]2013-05-23 17:27:33.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:33.667 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 486 (0x7f431bfce310) [ CSSD]2013-05-23 17:27:33.667 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce310 (RPC#486) state 6, flags 0x202 [ CSSD]2013-05-23 17:27:33.667 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce310, ret 0, client (nil) [ CSSD]2013-05-23 17:27:33.667 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 486 [ CSSD]2013-05-23 17:27:34.088 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:35.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:35.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:35.111 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:35.111 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:36.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:36.650 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b950) allocated [ CSSD]2013-05-23 17:27:36.651 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:27:36.652 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b950 - 1,2,3 [ CSSD]2013-05-23 17:27:36.652 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c083240) proc(0x7f431c13b950) pid(26744) version 11:1:1:4 [ CSSD]2013-05-23 17:27:36.657 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13b950), client(1/0x7f431c14e340) [ CSSD]2013-05-23 17:27:36.657 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14e340) [ CSSD]2013-05-23 17:27:36.657 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14e340 with con 0x7f431c13bb50, requested num -1 [ CSSD]2013-05-23 17:27:36.657 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:36.657 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 487 (0x7f431bfce390) [ CSSD]2013-05-23 17:27:36.658 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce390 (RPC#487) tag(1e7002d) sent to node 1 [ CSSD]2013-05-23 17:27:36.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01e7002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:36.658 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015248, gin 1, member 0 [ CSSD]2013-05-23 17:27:36.658 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e4b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:27:36.658 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:27:36.658 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:27:36.659 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce390 (RPC#487) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:36.659 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfce390, ret 0, client 0x7f431c14e340 member 0x7f431c13b840 [ CSSD]2013-05-23 17:27:36.659 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14e340 [ CSSD]2013-05-23 17:27:36.659 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 487 [ CSSD]2013-05-23 17:27:36.665 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14e340) [ CSSD]2013-05-23 17:27:36.665 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14e340), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:36.665 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:36.665 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 488 (0x7f431bfce410) [ CSSD]2013-05-23 17:27:36.665 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce410 (RPC#488) tag(1e8002d) sent to node 1 [ CSSD]2013-05-23 17:27:36.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01e8002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:36.666 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:36.666 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce410 (RPC#488) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:36.666 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce410, ret 0, client 0x7f431c14e340 member 0x7f431c13b840 [ CSSD]2013-05-23 17:27:36.666 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 488 [ CSSD]2013-05-23 17:27:36.666 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:27:36.868 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b950 [ CSSD]2013-05-23 17:27:36.868 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b950) con(0x7f431c083240) skgpid ospid 26744 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:37.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:37.071 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:38.127 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:39.077 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:39.077 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:40.114 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:40.115 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:40.132 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:41.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:41.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:41.986 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:41.986 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015249, gin 1, member 0 [ CSSD]2013-05-23 17:27:41.986 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:41.986 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:41.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:41.992 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:41.992 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:42.138 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:43.134 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:43.134 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:43.686 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:27:43.687 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 489 (0x7f431bfce490) [ CSSD]2013-05-23 17:27:43.687 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce490 (RPC#489) tag(1e9002d) sent to node 1 [ CSSD]2013-05-23 17:27:43.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:43.687 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 489 (0x7f431bfce490) [ CSSD]2013-05-23 17:27:43.687 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce490 (RPC#489) state 6, flags 0x202 [ CSSD]2013-05-23 17:27:43.687 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce490, ret 0, client (nil) [ CSSD]2013-05-23 17:27:43.687 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 489 [ CSSD]2013-05-23 17:27:43.820 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:43.820 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015250, gin 1, member 0 [ CSSD]2013-05-23 17:27:43.820 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:43.820 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:43.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:43.828 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:43.828 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:44.117 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:44.117 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:27:44.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:45.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:45.144 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:46.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:47.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:47.150 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:47.349 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13b880) allocated [ CSSD]2013-05-23 17:27:47.351 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:27:47.351 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13b880 - 1,2,3 [ CSSD]2013-05-23 17:27:47.351 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c083100) proc(0x7f431c13b880) pid(26904) version 11:1:1:4 [ CSSD]2013-05-23 17:27:47.358 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c13b880), client(1/0x7f431c13bf50) [ CSSD]2013-05-23 17:27:47.358 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13bf50) [ CSSD]2013-05-23 17:27:47.358 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13bf50 with con 0x7f431c083ad0, requested num -1 [ CSSD]2013-05-23 17:27:47.358 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:47.358 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 490 (0x7f431bfce510) [ CSSD]2013-05-23 17:27:47.358 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce510 (RPC#490) tag(1ea002d) sent to node 1 [ CSSD]2013-05-23 17:27:47.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01ea002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:47.358 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015251, gin 1, member 0 [ CSSD]2013-05-23 17:27:47.358 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14e410/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:27:47.359 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:27:47.359 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:27:47.359 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce510 (RPC#490) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:47.359 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfce510, ret 0, client 0x7f431c13bf50 member 0x7f431c13b770 [ CSSD]2013-05-23 17:27:47.359 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13bf50 [ CSSD]2013-05-23 17:27:47.359 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 490 [ CSSD]2013-05-23 17:27:47.365 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13bf50) [ CSSD]2013-05-23 17:27:47.365 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13bf50), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:47.365 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13b770) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:27:47.365 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 491 (0x7f431bfce590) [ CSSD]2013-05-23 17:27:47.365 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce590 (RPC#491) tag(1eb002d) sent to node 1 [ CSSD]2013-05-23 17:27:47.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01eb002d) incarnation(174631051) [ CSSD]2013-05-23 17:27:47.366 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:27:47.366 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce590 (RPC#491) state 6, flags 0x100 [ CSSD]2013-05-23 17:27:47.366 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce590, ret 0, client 0x7f431c13bf50 member 0x7f431c13b770 [ CSSD]2013-05-23 17:27:47.366 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 491 [ CSSD]2013-05-23 17:27:47.366 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13b770) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:27:47.568 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13b880 [ CSSD]2013-05-23 17:27:47.568 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13b880) con(0x7f431c083100) skgpid ospid 26904 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:48.206 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:49.120 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:49.120 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:49.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:49.177 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:50.217 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 17:27:50.217 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0cf720), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:27:50.217 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0cf720), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:27:50.217 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:27:50.221 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:27:50.221 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:27:50.221 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:27:50.222 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:27:50.222 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13f950 [ CSSD]2013-05-23 17:27:50.222 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13f950) con(0x7f431c0e6e20) skgpid 21601 ospid 21601 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:27:50.231 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:51.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:51.208 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:52.236 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:53.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:53.214 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:53.727 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:27:53.727 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 492 (0x7f431bfce610) [ CSSD]2013-05-23 17:27:53.727 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce610 (RPC#492) tag(1ec002d) sent to node 1 [ CSSD]2013-05-23 17:27:53.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:53.728 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 492 (0x7f431bfce610) [ CSSD]2013-05-23 17:27:53.728 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce610 (RPC#492) state 6, flags 0x202 [ CSSD]2013-05-23 17:27:53.728 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce610, ret 0, client (nil) [ CSSD]2013-05-23 17:27:53.728 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 492 [ CSSD]2013-05-23 17:27:54.123 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:54.123 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:54.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:55.225 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:55.225 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:56.296 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:57.231 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:57.231 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:27:57.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:57.241 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015252, gin 1, member 0 [ CSSD]2013-05-23 17:27:57.241 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:27:57.241 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:27:57.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:57.248 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:27:57.248 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:27:58.306 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:27:59.126 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:27:59.126 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:27:59.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:27:59.256 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:00.331 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:01.327 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:01.327 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:02.337 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:02.600 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:28:02.601 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:28:02.601 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:28:02.601 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(27055) version 11:1:1:4 [ CSSD]2013-05-23 17:28:02.618 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:28:02.618 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:02.618 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:28:02.618 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:02.618 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 493 (0x7f431bfce690) [ CSSD]2013-05-23 17:28:02.618 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce690 (RPC#493) tag(1ed002d) sent to node 1 [ CSSD]2013-05-23 17:28:02.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01ed002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:02.618 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015253, gin 1, member 0 [ CSSD]2013-05-23 17:28:02.618 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082b90/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:28:02.619 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:28:02.619 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:28:02.619 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce690 (RPC#493) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:02.619 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfce690, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:02.619 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:28:02.619 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 493 [ CSSD]2013-05-23 17:28:02.626 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:02.626 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:02.626 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:02.626 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 494 (0x7f431bfce710) [ CSSD]2013-05-23 17:28:02.626 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce710 (RPC#494) tag(1ee002d) sent to node 1 [ CSSD]2013-05-23 17:28:02.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01ee002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:02.626 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:02.626 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce710 (RPC#494) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:02.626 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce710, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:02.627 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 494 [ CSSD]2013-05-23 17:28:02.627 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:28:02.829 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:28:02.829 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 27055 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:28:03.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:03.334 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:03.757 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:28:03.757 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 495 (0x7f431bfce790) [ CSSD]2013-05-23 17:28:03.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce790 (RPC#495) tag(1ef002d) sent to node 1 [ CSSD]2013-05-23 17:28:03.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:03.758 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 495 (0x7f431bfce790) [ CSSD]2013-05-23 17:28:03.758 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce790 (RPC#495) state 6, flags 0x202 [ CSSD]2013-05-23 17:28:03.758 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce790, ret 0, client (nil) [ CSSD]2013-05-23 17:28:03.758 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 495 [ CSSD]2013-05-23 17:28:04.129 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:04.129 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:04.339 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:05.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:05.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:06.344 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:07.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:07.352 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:08.398 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:09.132 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:09.132 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:09.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:09.358 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:10.403 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:11.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:11.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:12.428 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:12.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:12.495 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015254, gin 1, member 0 [ CSSD]2013-05-23 17:28:12.495 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:28:12.495 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:28:12.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:12.502 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:28:12.502 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:28:13.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:13.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:13.788 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:28:13.788 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 496 (0x7f431bfce810) [ CSSD]2013-05-23 17:28:13.789 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce810 (RPC#496) tag(1f0002d) sent to node 1 [ CSSD]2013-05-23 17:28:13.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:13.789 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 496 (0x7f431bfce810) [ CSSD]2013-05-23 17:28:13.789 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce810 (RPC#496) state 6, flags 0x202 [ CSSD]2013-05-23 17:28:13.789 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce810, ret 0, client (nil) [ CSSD]2013-05-23 17:28:13.789 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 496 [ CSSD]2013-05-23 17:28:14.135 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:14.135 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:14.440 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:15.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:15.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:16.445 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:17.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:17.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:17.868 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:28:17.869 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:28:17.870 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:28:17.870 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c11fd30) pid(27216) version 11:1:1:4 [ CSSD]2013-05-23 17:28:17.876 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:28:17.876 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:17.876 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:28:17.876 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:17.876 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 497 (0x7f431bfce890) [ CSSD]2013-05-23 17:28:17.876 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce890 (RPC#497) tag(1f1002d) sent to node 1 [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01f1002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015255, gin 1, member 0 [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082b90/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce890 (RPC#497) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfce890, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:28:17.877 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 497 [ CSSD]2013-05-23 17:28:17.884 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:17.884 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:17.884 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:17.884 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 498 (0x7f431bfce910) [ CSSD]2013-05-23 17:28:17.884 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce910 (RPC#498) tag(1f2002d) sent to node 1 [ CSSD]2013-05-23 17:28:17.884 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01f2002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:17.884 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:17.884 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce910 (RPC#498) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:17.884 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfce910, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:17.884 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 498 [ CSSD]2013-05-23 17:28:17.885 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:28:18.087 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:28:18.087 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e6e20) skgpid ospid 27216 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:28:18.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:19.138 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:19.139 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:19.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:19.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:20.504 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:21.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:21.550 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:22.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:23.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:23.556 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:23.817 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:28:23.818 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 499 (0x7f431bfce990) [ CSSD]2013-05-23 17:28:23.818 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfce990 (RPC#499) tag(1f3002d) sent to node 1 [ CSSD]2013-05-23 17:28:23.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:23.819 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 499 (0x7f431bfce990) [ CSSD]2013-05-23 17:28:23.819 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfce990 (RPC#499) state 6, flags 0x202 [ CSSD]2013-05-23 17:28:23.819 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfce990, ret 0, client (nil) [ CSSD]2013-05-23 17:28:23.819 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 499 [ CSSD]2013-05-23 17:28:24.142 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:24.142 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:24.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:25.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:25.607 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:26.544 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:27.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:27.614 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:27.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:27.750 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015256, gin 1, member 0 [ CSSD]2013-05-23 17:28:27.750 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:28:27.750 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:28:27.757 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:27.757 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:28:27.757 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:28:28.144 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:28.144 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:28:28.599 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:29.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:29.625 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:30.605 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:31.631 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:31.631 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:32.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:33.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:33.047 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015257, gin 1, member 0 [ CSSD]2013-05-23 17:28:33.047 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:28:33.047 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:28:33.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:33.053 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:28:33.053 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:28:33.129 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:28:33.130 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:28:33.131 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:28:33.131 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(27358) version 11:1:1:4 [ CSSD]2013-05-23 17:28:33.136 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:28:33.136 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:33.136 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:28:33.136 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:33.136 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 500 (0x7f431bfcea10) [ CSSD]2013-05-23 17:28:33.136 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcea10 (RPC#500) tag(1f4002d) sent to node 1 [ CSSD]2013-05-23 17:28:33.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01f4002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:33.137 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015258, gin 1, member 0 [ CSSD]2013-05-23 17:28:33.137 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082b90/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:28:33.137 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:28:33.138 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:28:33.138 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcea10 (RPC#500) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:33.138 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcea10, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:33.138 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:28:33.138 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 500 [ CSSD]2013-05-23 17:28:33.144 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:33.144 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:33.144 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:33.144 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 501 (0x7f431bfcea90) [ CSSD]2013-05-23 17:28:33.144 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcea90 (RPC#501) tag(1f5002d) sent to node 1 [ CSSD]2013-05-23 17:28:33.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01f5002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:33.145 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:33.145 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcea90 (RPC#501) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:33.145 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcea90, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:33.145 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 501 [ CSSD]2013-05-23 17:28:33.145 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:28:33.147 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:33.148 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:33.347 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:28:33.347 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 27358 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:28:33.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:33.657 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:33.837 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:28:33.838 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 502 (0x7f431bfceb10) [ CSSD]2013-05-23 17:28:33.838 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfceb10 (RPC#502) tag(1f6002d) sent to node 1 [ CSSD]2013-05-23 17:28:33.839 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:33.839 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 502 (0x7f431bfceb10) [ CSSD]2013-05-23 17:28:33.839 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfceb10 (RPC#502) state 6, flags 0x202 [ CSSD]2013-05-23 17:28:33.839 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfceb10, ret 0, client (nil) [ CSSD]2013-05-23 17:28:33.839 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 502 [ CSSD]2013-05-23 17:28:34.620 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:35.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:35.708 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:36.626 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:36.910 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:28:36.911 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:28:36.911 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:28:36.911 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c11fd30) pid(27425) version 11:1:1:4 [ CSSD]2013-05-23 17:28:36.927 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:28:36.927 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:36.927 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:28:36.928 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:36.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 503 (0x7f431bfceb90) [ CSSD]2013-05-23 17:28:36.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfceb90 (RPC#503) tag(1f7002d) sent to node 1 [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01f7002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015259, gin 1, member 0 [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082b90/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfceb90 (RPC#503) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfceb90, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:36.928 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:28:36.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 503 [ CSSD]2013-05-23 17:28:36.936 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:28:36.936 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:36.936 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:36.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 504 (0x7f431bfcec10) [ CSSD]2013-05-23 17:28:36.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcec10 (RPC#504) tag(1f8002d) sent to node 1 [ CSSD]2013-05-23 17:28:36.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01f8002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:36.936 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:36.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcec10 (RPC#504) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:36.937 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcec10, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:28:36.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 504 [ CSSD]2013-05-23 17:28:36.937 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:28:37.139 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:28:37.139 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e6e20) skgpid ospid 27425 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:28:37.714 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:37.714 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:38.150 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:38.150 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:38.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:39.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:39.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:40.645 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:41.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:41.732 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:42.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:42.556 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015260, gin 1, member 0 [ CSSD]2013-05-23 17:28:42.556 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:28:42.556 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:28:42.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:42.563 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:28:42.563 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:28:42.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:42.686 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:28:42.686 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:28:42.688 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:28:42.688 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:28:42.689 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:28:42.689 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:28:43.011 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:43.011 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015261, gin 1, member 0 [ CSSD]2013-05-23 17:28:43.011 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:28:43.011 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:28:43.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:43.020 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:28:43.020 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:28:43.153 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:43.154 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:43.737 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:43.737 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:43.858 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:28:43.858 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 505 (0x7f431bfcec90) [ CSSD]2013-05-23 17:28:43.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcec90 (RPC#505) tag(1f9002d) sent to node 1 [ CSSD]2013-05-23 17:28:43.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:43.859 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 505 (0x7f431bfcec90) [ CSSD]2013-05-23 17:28:43.859 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcec90 (RPC#505) state 6, flags 0x202 [ CSSD]2013-05-23 17:28:43.859 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcec90, ret 0, client (nil) [ CSSD]2013-05-23 17:28:43.859 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 505 [ CSSD]2013-05-23 17:28:44.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:45.747 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:45.747 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:46.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:47.156 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:47.156 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:28:47.753 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:47.753 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:48.381 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:28:48.382 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:28:48.382 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:28:48.382 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(27531) version 11:1:1:4 [ CSSD]2013-05-23 17:28:48.389 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 17:28:48.389 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:28:48.389 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 17:28:48.389 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:48.389 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 506 (0x7f431bfced10) [ CSSD]2013-05-23 17:28:48.389 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfced10 (RPC#506) tag(1fa002d) sent to node 1 [ CSSD]2013-05-23 17:28:48.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01fa002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:48.389 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015262, gin 1, member 0 [ CSSD]2013-05-23 17:28:48.389 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:28:48.390 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:28:48.390 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:28:48.390 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfced10 (RPC#506) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:48.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfced10, ret 0, client 0x2312950 member 0x231d6c0 [ CSSD]2013-05-23 17:28:48.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:28:48.390 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 506 [ CSSD]2013-05-23 17:28:48.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:28:48.396 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:48.396 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231d6c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:28:48.396 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 507 (0x7f431bfced90) [ CSSD]2013-05-23 17:28:48.396 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfced90 (RPC#507) tag(1fb002d) sent to node 1 [ CSSD]2013-05-23 17:28:48.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01fb002d) incarnation(174631051) [ CSSD]2013-05-23 17:28:48.397 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:28:48.397 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfced90 (RPC#507) state 6, flags 0x100 [ CSSD]2013-05-23 17:28:48.397 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfced90, ret 0, client 0x2312950 member 0x231d6c0 [ CSSD]2013-05-23 17:28:48.397 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 507 [ CSSD]2013-05-23 17:28:48.397 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231d6c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:28:48.599 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:28:48.599 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 27531 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:28:48.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:49.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:49.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:50.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:51.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:51.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:52.159 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:52.159 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:52.730 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:53.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:53.836 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:53.898 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:28:53.899 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 508 (0x7f431bfcee10) [ CSSD]2013-05-23 17:28:53.899 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcee10 (RPC#508) tag(1fc002d) sent to node 1 [ CSSD]2013-05-23 17:28:53.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:53.899 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 508 (0x7f431bfcee10) [ CSSD]2013-05-23 17:28:53.899 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcee10 (RPC#508) state 6, flags 0x202 [ CSSD]2013-05-23 17:28:53.899 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcee10, ret 0, client (nil) [ CSSD]2013-05-23 17:28:53.900 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 508 [ CSSD]2013-05-23 17:28:54.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:54.943 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c134bc0) [ CSSD]2013-05-23 17:28:54.943 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c134bc0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:28:54.943 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c134bc0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:28:54.943 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 17:28:54.947 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:28:54.947 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:28:54.947 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:28:54.947 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:28:54.947 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 17:28:54.947 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c108e60) skgpid 22271 ospid 22271 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:28:55.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:55.848 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:56.747 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:57.162 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:28:57.163 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:28:57.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:57.854 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:28:58.263 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:58.263 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015263, gin 1, member 0 [ CSSD]2013-05-23 17:28:58.263 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:28:58.263 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:28:58.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:58.270 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:28:58.270 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:28:58.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:28:59.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:28:59.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:00.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:01.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:01.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:02.165 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:02.165 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:02.829 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:03.631 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 17:29:03.632 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:29:03.632 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 17:29:03.632 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22d7cd0) pid(27689) version 11:1:1:4 [ CSSD]2013-05-23 17:29:03.648 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7cd0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:29:03.648 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:29:03.648 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c108b60, requested num -1 [ CSSD]2013-05-23 17:29:03.648 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:03.648 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 509 (0x7f431bfcee90) [ CSSD]2013-05-23 17:29:03.648 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcee90 (RPC#509) tag(1fd002d) sent to node 1 [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(01fd002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015264, gin 1, member 0 [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcee90 (RPC#509) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcee90, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:29:03.649 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 509 [ CSSD]2013-05-23 17:29:03.657 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:29:03.657 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:03.657 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:03.657 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 510 (0x7f431bfcef10) [ CSSD]2013-05-23 17:29:03.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcef10 (RPC#510) tag(1fe002d) sent to node 1 [ CSSD]2013-05-23 17:29:03.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(01fe002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:03.657 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:03.657 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcef10 (RPC#510) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:03.657 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcef10, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:29:03.657 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 510 [ CSSD]2013-05-23 17:29:03.657 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:29:03.860 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 17:29:03.860 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2312880) skgpid ospid 27689 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:29:03.907 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:03.907 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:03.918 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:29:03.919 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 511 (0x7f431bfcef90) [ CSSD]2013-05-23 17:29:03.919 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcef90 (RPC#511) tag(1ff002d) sent to node 1 [ CSSD]2013-05-23 17:29:03.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:03.919 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 511 (0x7f431bfcef90) [ CSSD]2013-05-23 17:29:03.919 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcef90 (RPC#511) state 6, flags 0x202 [ CSSD]2013-05-23 17:29:03.919 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcef90, ret 0, client (nil) [ CSSD]2013-05-23 17:29:03.920 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 511 [ CSSD]2013-05-23 17:29:04.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:05.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:05.919 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:06.847 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:07.168 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:07.168 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:07.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:07.925 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:08.869 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:09.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:09.936 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:10.875 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:11.170 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:11.170 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:29:11.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:11.962 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:12.881 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:13.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:13.514 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015265, gin 1, member 0 [ CSSD]2013-05-23 17:29:13.514 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:29:13.514 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:29:13.521 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:13.521 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:29:13.521 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:29:13.948 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:29:13.949 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 512 (0x7f431bfcf010) [ CSSD]2013-05-23 17:29:13.949 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf010 (RPC#512) tag(200002d) sent to node 1 [ CSSD]2013-05-23 17:29:13.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:13.949 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 512 (0x7f431bfcf010) [ CSSD]2013-05-23 17:29:13.949 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf010 (RPC#512) state 6, flags 0x202 [ CSSD]2013-05-23 17:29:13.949 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf010, ret 0, client (nil) [ CSSD]2013-05-23 17:29:13.950 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 512 [ CSSD]2013-05-23 17:29:13.967 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:13.967 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:14.922 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:16.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:16.019 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:16.173 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:16.173 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:16.927 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:18.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:18.025 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:18.895 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 17:29:18.896 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:29:18.897 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 17:29:18.897 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108b60) proc(0x7f431c134ba0) pid(27847) version 11:1:1:4 [ CSSD]2013-05-23 17:29:18.903 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:29:18.903 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:29:18.903 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:29:18.903 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:18.903 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 513 (0x7f431bfcf090) [ CSSD]2013-05-23 17:29:18.903 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf090 (RPC#513) tag(201002d) sent to node 1 [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0201002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015266, gin 1, member 0 [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf090 (RPC#513) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcf090, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:29:18.904 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 513 [ CSSD]2013-05-23 17:29:18.910 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:29:18.910 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:18.910 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:18.910 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 514 (0x7f431bfcf110) [ CSSD]2013-05-23 17:29:18.910 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf110 (RPC#514) tag(202002d) sent to node 1 [ CSSD]2013-05-23 17:29:18.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0202002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:18.911 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:18.911 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf110 (RPC#514) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:18.911 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcf110, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:18.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 514 [ CSSD]2013-05-23 17:29:18.911 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:29:18.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:19.113 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 17:29:19.113 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c108b60) skgpid ospid 27847 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:29:20.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:20.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:20.945 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:21.175 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:21.175 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:22.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:22.042 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:22.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:23.979 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:29:23.979 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 515 (0x7f431bfcf190) [ CSSD]2013-05-23 17:29:23.979 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf190 (RPC#515) tag(203002d) sent to node 1 [ CSSD]2013-05-23 17:29:23.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:23.979 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 515 (0x7f431bfcf190) [ CSSD]2013-05-23 17:29:23.979 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf190 (RPC#515) state 6, flags 0x202 [ CSSD]2013-05-23 17:29:23.979 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf190, ret 0, client (nil) [ CSSD]2013-05-23 17:29:23.980 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 515 [ CSSD]2013-05-23 17:29:24.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:24.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:24.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:26.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:26.140 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:26.178 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:26.178 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:26.968 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:28.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:28.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:28.764 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:28.764 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015267, gin 1, member 0 [ CSSD]2013-05-23 17:29:28.764 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:29:28.764 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:29:28.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:28.771 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:29:28.771 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:29:29.021 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:29.848 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:29:29.849 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:29:29.850 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:29:29.851 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:29:29.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:29.931 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015268, gin 1, member 0 [ CSSD]2013-05-23 17:29:29.931 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:29:29.931 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:29:29.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:29.937 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:29:29.937 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:29:30.218 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:30.218 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:31.026 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:31.181 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:31.181 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:32.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:32.224 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:33.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:33.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:33.299 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015269, gin 1, member 0 [ CSSD]2013-05-23 17:29:33.299 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:29:33.299 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:29:33.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:33.306 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:29:33.306 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:29:34.009 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:29:34.009 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 516 (0x7f431bfcf210) [ CSSD]2013-05-23 17:29:34.009 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf210 (RPC#516) tag(204002d) sent to node 1 [ CSSD]2013-05-23 17:29:34.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:34.010 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 516 (0x7f431bfcf210) [ CSSD]2013-05-23 17:29:34.010 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf210 (RPC#516) state 6, flags 0x202 [ CSSD]2013-05-23 17:29:34.010 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf210, ret 0, client (nil) [ CSSD]2013-05-23 17:29:34.010 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 516 [ CSSD]2013-05-23 17:29:34.146 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 17:29:34.147 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:29:34.148 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 17:29:34.148 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108b60) proc(0x7f431c134ba0) pid(27988) version 11:1:1:4 [ CSSD]2013-05-23 17:29:34.153 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:29:34.153 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:29:34.153 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:29:34.153 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:34.153 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 517 (0x7f431bfcf290) [ CSSD]2013-05-23 17:29:34.154 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf290 (RPC#517) tag(205002d) sent to node 1 [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0205002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015270, gin 1, member 0 [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf290 (RPC#517) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcf290, ret 0, client 0x7f431c109f80 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:29:34.154 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 517 [ CSSD]2013-05-23 17:29:34.160 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:29:34.160 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:34.160 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:34.160 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 518 (0x7f431bfcf310) [ CSSD]2013-05-23 17:29:34.160 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf310 (RPC#518) tag(206002d) sent to node 1 [ CSSD]2013-05-23 17:29:34.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0206002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:34.161 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:34.161 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf310 (RPC#518) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:34.161 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcf310, ret 0, client 0x7f431c109f80 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:34.161 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 518 [ CSSD]2013-05-23 17:29:34.161 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:29:34.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:34.230 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:34.363 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 17:29:34.363 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c108b60) skgpid ospid 27988 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:29:35.044 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:35.184 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:35.184 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:29:36.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:36.240 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:37.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:37.174 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 17:29:37.175 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:29:37.176 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 17:29:37.176 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134ba0) pid(28050) version 11:1:1:4 [ CSSD]2013-05-23 17:29:37.192 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:29:37.192 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:29:37.192 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:29:37.192 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:37.192 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 519 (0x7f431bfcf390) [ CSSD]2013-05-23 17:29:37.192 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf390 (RPC#519) tag(207002d) sent to node 1 [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0207002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015271, gin 1, member 0 [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf390 (RPC#519) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcf390, ret 0, client 0x7f431c0e5910 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:29:37.193 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 519 [ CSSD]2013-05-23 17:29:37.201 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:29:37.201 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:37.201 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:37.201 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 520 (0x7f431bfcf410) [ CSSD]2013-05-23 17:29:37.201 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf410 (RPC#520) tag(208002d) sent to node 1 [ CSSD]2013-05-23 17:29:37.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0208002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:37.201 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:37.201 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf410 (RPC#520) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:37.201 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcf410, ret 0, client 0x7f431c0e5910 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:37.202 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 520 [ CSSD]2013-05-23 17:29:37.202 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:29:37.404 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 17:29:37.404 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c108e60) skgpid ospid 28050 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:29:38.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:38.266 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:39.122 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:40.187 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:40.187 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:40.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:40.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:41.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:42.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:42.324 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:43.158 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:44.019 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:44.019 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015272, gin 1, member 0 [ CSSD]2013-05-23 17:29:44.019 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:29:44.019 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:29:44.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:44.026 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:29:44.026 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:29:44.029 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:29:44.029 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 521 (0x7f431bfcf490) [ CSSD]2013-05-23 17:29:44.029 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf490 (RPC#521) tag(209002d) sent to node 1 [ CSSD]2013-05-23 17:29:44.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:44.030 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 521 (0x7f431bfcf490) [ CSSD]2013-05-23 17:29:44.030 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf490 (RPC#521) state 6, flags 0x202 [ CSSD]2013-05-23 17:29:44.030 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf490, ret 0, client (nil) [ CSSD]2013-05-23 17:29:44.030 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 521 [ CSSD]2013-05-23 17:29:44.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:44.330 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:45.164 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:45.190 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:45.190 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:46.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:46.360 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:47.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:48.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:48.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:49.222 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:49.395 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 17:29:49.396 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:29:49.397 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 17:29:49.397 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c134ba0) pid(28151) version 11:1:1:4 [ CSSD]2013-05-23 17:29:49.402 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:29:49.402 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:29:49.402 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:29:49.402 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:49.402 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 522 (0x7f431bfcf510) [ CSSD]2013-05-23 17:29:49.403 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf510 (RPC#522) tag(20a002d) sent to node 1 [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(020a002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015273, gin 1, member 0 [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf510 (RPC#522) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcf510, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:49.403 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:29:49.404 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 522 [ CSSD]2013-05-23 17:29:49.410 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:29:49.410 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:49.410 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:29:49.410 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 523 (0x7f431bfcf590) [ CSSD]2013-05-23 17:29:49.410 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf590 (RPC#523) tag(20b002d) sent to node 1 [ CSSD]2013-05-23 17:29:49.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(020b002d) incarnation(174631051) [ CSSD]2013-05-23 17:29:49.411 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:29:49.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf590 (RPC#523) state 6, flags 0x100 [ CSSD]2013-05-23 17:29:49.411 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcf590, ret 0, client 0x7f431c0cf720 member 0x7f431c13f950 [ CSSD]2013-05-23 17:29:49.411 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 523 [ CSSD]2013-05-23 17:29:49.411 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:29:49.613 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 17:29:49.613 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c108e60) skgpid ospid 28151 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:29:50.193 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:50.193 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:50.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:50.419 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:51.228 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:52.425 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:52.425 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:53.239 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:54.050 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:29:54.050 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 524 (0x7f431bfcf610) [ CSSD]2013-05-23 17:29:54.050 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf610 (RPC#524) tag(20c002d) sent to node 1 [ CSSD]2013-05-23 17:29:54.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:54.051 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 524 (0x7f431bfcf610) [ CSSD]2013-05-23 17:29:54.051 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf610 (RPC#524) state 6, flags 0x202 [ CSSD]2013-05-23 17:29:54.051 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf610, ret 0, client (nil) [ CSSD]2013-05-23 17:29:54.051 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 524 [ CSSD]2013-05-23 17:29:54.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:54.450 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:55.196 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:29:55.196 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:29:55.244 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:56.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:56.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:57.250 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:58.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:58.468 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:29:59.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:29:59.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:59.270 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015274, gin 1, member 0 [ CSSD]2013-05-23 17:29:59.270 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:29:59.270 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:29:59.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:29:59.277 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:29:59.277 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:30:00.200 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:00.200 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:00.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:00.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:01.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:02.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:02.525 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:03.320 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:04.070 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:30:04.070 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 525 (0x7f431bfcf690) [ CSSD]2013-05-23 17:30:04.070 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf690 (RPC#525) tag(20d002d) sent to node 1 [ CSSD]2013-05-23 17:30:04.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:04.071 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 525 (0x7f431bfcf690) [ CSSD]2013-05-23 17:30:04.071 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf690 (RPC#525) state 6, flags 0x202 [ CSSD]2013-05-23 17:30:04.071 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf690, ret 0, client (nil) [ CSSD]2013-05-23 17:30:04.071 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 525 [ CSSD]2013-05-23 17:30:04.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:04.551 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:04.644 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 17:30:04.645 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:30:04.646 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 17:30:04.646 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134ba0) pid(28471) version 11:1:1:4 [ CSSD]2013-05-23 17:30:04.652 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:30:04.652 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:30:04.652 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:30:04.652 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:04.652 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 526 (0x7f431bfcf710) [ CSSD]2013-05-23 17:30:04.652 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf710 (RPC#526) tag(20e002d) sent to node 1 [ CSSD]2013-05-23 17:30:04.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(020e002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:04.652 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015275, gin 1, member 0 [ CSSD]2013-05-23 17:30:04.652 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:30:04.653 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:30:04.653 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:30:04.653 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf710 (RPC#526) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:04.653 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcf710, ret 0, client 0x7f431c109f80 member 0x7f431c13f950 [ CSSD]2013-05-23 17:30:04.653 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:30:04.653 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 526 [ CSSD]2013-05-23 17:30:04.659 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:30:04.659 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:04.659 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:04.659 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 527 (0x7f431bfcf790) [ CSSD]2013-05-23 17:30:04.659 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf790 (RPC#527) tag(20f002d) sent to node 1 [ CSSD]2013-05-23 17:30:04.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(020f002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:04.660 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:04.660 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf790 (RPC#527) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:04.660 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcf790, ret 0, client 0x7f431c109f80 member 0x7f431c13f950 [ CSSD]2013-05-23 17:30:04.660 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 527 [ CSSD]2013-05-23 17:30:04.660 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:30:04.862 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 17:30:04.862 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c11ab50) skgpid ospid 28471 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:05.203 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:05.203 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:05.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:06.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:06.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:07.351 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:07.993 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 17:30:07.994 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:30:07.995 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 17:30:07.995 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134ba0) pid(28547) version 11:1:1:4 [ CSSD]2013-05-23 17:30:08.184 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b850) allocated [ CSSD]2013-05-23 17:30:08.185 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:30:08.186 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b850 - 1,2,3 [ CSSD]2013-05-23 17:30:08.186 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c12b850) pid(27624/27624) version 11:1:1:4 [ CSSD]2013-05-23 17:30:08.192 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c12b850), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:30:08.192 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:30:08.192 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:30:08.192 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:30:08.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:08.567 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:08.612 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 17:30:08.612 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c11ab50) skgpid ospid 28547 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:09.362 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:10.206 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:10.206 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:10.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:10.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:11.368 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:12.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:12.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:13.420 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:14.130 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:30:14.130 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 528 (0x7f431bfcf810) [ CSSD]2013-05-23 17:30:14.131 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf810 (RPC#528) tag(210002d) sent to node 1 [ CSSD]2013-05-23 17:30:14.131 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:14.131 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 528 (0x7f431bfcf810) [ CSSD]2013-05-23 17:30:14.131 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf810 (RPC#528) state 6, flags 0x202 [ CSSD]2013-05-23 17:30:14.131 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf810, ret 0, client (nil) [ CSSD]2013-05-23 17:30:14.131 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 528 [ CSSD]2013-05-23 17:30:14.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:14.520 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015276, gin 1, member 0 [ CSSD]2013-05-23 17:30:14.520 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:30:14.520 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:30:14.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:14.527 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:30:14.527 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:30:14.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:14.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:15.208 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:15.209 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:15.426 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:16.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:16.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:17.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:18.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:18.667 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:19.210 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:19.211 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:30:19.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:19.894 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:30:19.895 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:30:19.895 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:30:19.895 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c11fd30) pid(28737) version 11:1:1:4 [ CSSD]2013-05-23 17:30:19.901 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c13f730) [ CSSD]2013-05-23 17:30:19.901 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:30:19.901 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c108b60, requested num -1 [ CSSD]2013-05-23 17:30:19.901 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:19.901 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 529 (0x7f431bfcf890) [ CSSD]2013-05-23 17:30:19.901 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf890 (RPC#529) tag(211002d) sent to node 1 [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0211002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015277, gin 1, member 0 [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf890 (RPC#529) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcf890, ret 0, client 0x7f431c13f730 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:19.902 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 17:30:19.903 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 529 [ CSSD]2013-05-23 17:30:19.909 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:30:19.909 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:19.909 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:19.909 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 530 (0x7f431bfcf910) [ CSSD]2013-05-23 17:30:19.909 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf910 (RPC#530) tag(212002d) sent to node 1 [ CSSD]2013-05-23 17:30:19.910 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0212002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:19.910 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:19.910 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf910 (RPC#530) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:19.910 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcf910, ret 0, client 0x7f431c13f730 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:19.910 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 530 [ CSSD]2013-05-23 17:30:19.910 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:30:20.113 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:30:20.113 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c11ab50) skgpid ospid 28737 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:20.669 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:20.669 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:21.468 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:22.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:22.675 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:23.521 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:24.160 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:30:24.160 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 531 (0x7f431bfcf990) [ CSSD]2013-05-23 17:30:24.160 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcf990 (RPC#531) tag(213002d) sent to node 1 [ CSSD]2013-05-23 17:30:24.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:24.161 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 531 (0x7f431bfcf990) [ CSSD]2013-05-23 17:30:24.161 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcf990 (RPC#531) state 6, flags 0x202 [ CSSD]2013-05-23 17:30:24.161 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcf990, ret 0, client (nil) [ CSSD]2013-05-23 17:30:24.161 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 531 [ CSSD]2013-05-23 17:30:24.213 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:24.213 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:24.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:24.728 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:25.547 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:26.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:26.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:27.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:28.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:28.779 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:29.216 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:29.216 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:29.574 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:29.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:29.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015278, gin 1, member 0 [ CSSD]2013-05-23 17:30:29.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:30:29.770 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:30:29.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:29.776 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:30:29.776 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:30:30.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:30.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:31.580 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:32.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:32.836 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:33.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:33.560 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015279, gin 1, member 0 [ CSSD]2013-05-23 17:30:33.560 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:30:33.560 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:30:33.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:33.567 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:30:33.567 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:30:33.631 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:34.190 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:30:34.190 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 532 (0x7f431bfcfa10) [ CSSD]2013-05-23 17:30:34.190 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfa10 (RPC#532) tag(214002d) sent to node 1 [ CSSD]2013-05-23 17:30:34.191 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:34.191 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 532 (0x7f431bfcfa10) [ CSSD]2013-05-23 17:30:34.191 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfa10 (RPC#532) state 6, flags 0x202 [ CSSD]2013-05-23 17:30:34.191 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcfa10, ret 0, client (nil) [ CSSD]2013-05-23 17:30:34.191 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 532 [ CSSD]2013-05-23 17:30:34.219 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:34.219 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:34.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:34.848 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:35.144 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:30:35.145 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:30:35.146 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:30:35.146 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(28891) version 11:1:1:4 [ CSSD]2013-05-23 17:30:35.151 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:30:35.151 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:30:35.151 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:30:35.151 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:35.151 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 533 (0x7f431bfcfa90) [ CSSD]2013-05-23 17:30:35.152 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfa90 (RPC#533) tag(215002d) sent to node 1 [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0215002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015280, gin 1, member 0 [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfa90 (RPC#533) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcfa90, ret 0, client 0x7f431c0cf720 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:35.152 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:30:35.153 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 533 [ CSSD]2013-05-23 17:30:35.158 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:30:35.158 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:35.158 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:35.158 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 534 (0x7f431bfcfb10) [ CSSD]2013-05-23 17:30:35.158 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfb10 (RPC#534) tag(216002d) sent to node 1 [ CSSD]2013-05-23 17:30:35.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0216002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:35.159 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:35.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfb10 (RPC#534) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:35.159 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcfb10, ret 0, client 0x7f431c0cf720 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:35.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 534 [ CSSD]2013-05-23 17:30:35.159 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:30:35.361 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:30:35.361 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 28891 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:35.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:36.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:36.853 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:37.442 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:30:37.443 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:30:37.443 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:30:37.443 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(28959) version 11:1:1:4 [ CSSD]2013-05-23 17:30:37.449 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:30:37.449 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:30:37.449 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:30:37.449 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:37.449 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 535 (0x7f431bfcfb90) [ CSSD]2013-05-23 17:30:37.450 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfb90 (RPC#535) tag(217002d) sent to node 1 [ CSSD]2013-05-23 17:30:37.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0217002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:37.450 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015281, gin 1, member 0 [ CSSD]2013-05-23 17:30:37.450 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:30:37.451 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:30:37.451 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:30:37.451 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfb90 (RPC#535) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:37.451 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcfb90, ret 0, client 0x7f431c0e5910 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:37.451 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:30:37.451 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 535 [ CSSD]2013-05-23 17:30:37.458 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:30:37.458 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:37.458 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:37.458 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 536 (0x7f431bfcfc10) [ CSSD]2013-05-23 17:30:37.458 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfc10 (RPC#536) tag(218002d) sent to node 1 [ CSSD]2013-05-23 17:30:37.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0218002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:37.458 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:37.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfc10 (RPC#536) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:37.458 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcfc10, ret 0, client 0x7f431c0e5910 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:37.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 536 [ CSSD]2013-05-23 17:30:37.458 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:30:37.642 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:37.661 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:30:37.661 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 28959 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:38.878 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:38.878 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:39.222 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:39.222 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:39.653 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:40.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:40.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:41.659 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:42.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:42.937 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:43.672 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:44.225 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:44.225 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:44.233 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:30:44.233 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 537 (0x7f431bfcfc90) [ CSSD]2013-05-23 17:30:44.234 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfc90 (RPC#537) tag(219002d) sent to node 1 [ CSSD]2013-05-23 17:30:44.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:44.234 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 537 (0x7f431bfcfc90) [ CSSD]2013-05-23 17:30:44.234 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfc90 (RPC#537) state 6, flags 0x202 [ CSSD]2013-05-23 17:30:44.234 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcfc90, ret 0, client (nil) [ CSSD]2013-05-23 17:30:44.234 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 537 [ CSSD]2013-05-23 17:30:44.948 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:44.948 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:45.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:45.020 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015282, gin 1, member 0 [ CSSD]2013-05-23 17:30:45.020 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:30:45.020 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:30:45.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:45.026 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:30:45.026 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:30:45.678 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:46.954 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:46.954 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:47.684 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:48.960 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:48.960 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:49.228 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:49.228 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:49.735 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:50.392 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:30:50.393 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:30:50.394 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:30:50.394 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c11fd30) pid(29066) version 11:1:1:4 [ CSSD]2013-05-23 17:30:50.399 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c13f730) [ CSSD]2013-05-23 17:30:50.399 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:30:50.399 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:30:50.399 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:50.399 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 538 (0x7f431bfcfd10) [ CSSD]2013-05-23 17:30:50.399 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfd10 (RPC#538) tag(21a002d) sent to node 1 [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(021a002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015283, gin 1, member 0 [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfd10 (RPC#538) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcfd10, ret 0, client 0x7f431c13f730 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 17:30:50.400 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 538 [ CSSD]2013-05-23 17:30:50.406 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:30:50.406 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:50.406 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:50.406 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 539 (0x7f431bfcfd90) [ CSSD]2013-05-23 17:30:50.406 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfd90 (RPC#539) tag(21b002d) sent to node 1 [ CSSD]2013-05-23 17:30:50.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(021b002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:50.407 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:50.407 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfd90 (RPC#539) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:50.407 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcfd90, ret 0, client 0x7f431c13f730 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:50.407 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 539 [ CSSD]2013-05-23 17:30:50.407 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:30:50.609 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:30:50.609 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c108e60) skgpid ospid 29066 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:50.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:50.972 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:51.741 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:52.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:52.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:53.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:54.231 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:54.231 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:54.261 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:30:54.261 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 540 (0x7f431bfcfe10) [ CSSD]2013-05-23 17:30:54.261 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfe10 (RPC#540) tag(21c002d) sent to node 1 [ CSSD]2013-05-23 17:30:54.262 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:54.262 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 540 (0x7f431bfcfe10) [ CSSD]2013-05-23 17:30:54.262 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfe10 (RPC#540) state 6, flags 0x202 [ CSSD]2013-05-23 17:30:54.262 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcfe10, ret 0, client (nil) [ CSSD]2013-05-23 17:30:54.262 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 540 [ CSSD]2013-05-23 17:30:55.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:55.029 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:55.778 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:57.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:57.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:57.785 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:30:59.060 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:30:59.061 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:30:59.062 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:30:59.062 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c11fd30) pid(29170) version 11:1:1:4 [ CSSD]2013-05-23 17:30:59.079 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:30:59.079 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:30:59.079 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:30:59.079 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:59.079 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 541 (0x7f431bfcfe90) [ CSSD]2013-05-23 17:30:59.079 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcfe90 (RPC#541) tag(21d002d) sent to node 1 [ CSSD]2013-05-23 17:30:59.079 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(021d002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:59.079 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015284, gin 1, member 0 [ CSSD]2013-05-23 17:30:59.079 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:30:59.080 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:30:59.080 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:30:59.080 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcfe90 (RPC#541) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:59.080 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfcfe90, ret 0, client 0x7f431c0cf720 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:59.080 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:30:59.080 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 541 [ CSSD]2013-05-23 17:30:59.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:30:59.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:30:59.087 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:30:59.087 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:59.087 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:30:59.087 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 542 (0x7f431bfcff10) [ CSSD]2013-05-23 17:30:59.087 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcff10 (RPC#542) tag(21e002d) sent to node 1 [ CSSD]2013-05-23 17:30:59.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(021e002d) incarnation(174631051) [ CSSD]2013-05-23 17:30:59.087 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:30:59.087 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcff10 (RPC#542) state 6, flags 0x100 [ CSSD]2013-05-23 17:30:59.087 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfcff10, ret 0, client 0x7f431c0cf720 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:30:59.087 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 542 [ CSSD]2013-05-23 17:30:59.087 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:30:59.234 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:30:59.235 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:30:59.290 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:30:59.290 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c108e60) skgpid ospid 29170 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:30:59.835 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:00.269 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:00.269 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015285, gin 1, member 0 [ CSSD]2013-05-23 17:31:00.269 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:31:00.270 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:31:00.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:00.276 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:31:00.276 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:31:01.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:01.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:01.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:03.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:03.138 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:03.237 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:03.237 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:31:03.853 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:04.321 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:31:04.321 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 543 (0x7f431bfcff90) [ CSSD]2013-05-23 17:31:04.322 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfcff90 (RPC#543) tag(21f002d) sent to node 1 [ CSSD]2013-05-23 17:31:04.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:04.322 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 543 (0x7f431bfcff90) [ CSSD]2013-05-23 17:31:04.322 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfcff90 (RPC#543) state 6, flags 0x202 [ CSSD]2013-05-23 17:31:04.322 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfcff90, ret 0, client (nil) [ CSSD]2013-05-23 17:31:04.322 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 543 [ CSSD]2013-05-23 17:31:05.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:05.149 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:05.641 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:31:05.642 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:31:05.642 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:31:05.642 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(29251) version 11:1:1:4 [ CSSD]2013-05-23 17:31:05.648 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:31:05.648 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:31:05.648 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:31:05.648 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:05.648 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 544 (0x7f431bfd0010) [ CSSD]2013-05-23 17:31:05.648 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0010 (RPC#544) tag(220002d) sent to node 1 [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0220002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015286, gin 1, member 0 [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0010 (RPC#544) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0010, ret 0, client 0x7f431c0e5910 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:31:05.649 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 544 [ CSSD]2013-05-23 17:31:05.655 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:31:05.655 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:05.655 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:05.655 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 545 (0x7f431bfd0090) [ CSSD]2013-05-23 17:31:05.655 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0090 (RPC#545) tag(221002d) sent to node 1 [ CSSD]2013-05-23 17:31:05.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0221002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:05.656 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:05.656 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0090 (RPC#545) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:05.656 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0090, ret 0, client 0x7f431c0e5910 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:31:05.656 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 545 [ CSSD]2013-05-23 17:31:05.656 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:31:05.858 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:05.858 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 29251 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:05.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:07.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:07.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:07.904 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:08.240 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:08.240 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:09.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:09.221 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:09.955 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:11.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:11.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:11.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:13.240 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:13.240 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:13.243 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:13.243 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:14.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:14.351 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:31:14.351 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 546 (0x7f431bfd0110) [ CSSD]2013-05-23 17:31:14.351 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0110 (RPC#546) tag(222002d) sent to node 1 [ CSSD]2013-05-23 17:31:14.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:14.352 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 546 (0x7f431bfd0110) [ CSSD]2013-05-23 17:31:14.352 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0110 (RPC#546) state 6, flags 0x202 [ CSSD]2013-05-23 17:31:14.352 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0110, ret 0, client (nil) [ CSSD]2013-05-23 17:31:14.352 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 546 [ CSSD]2013-05-23 17:31:15.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:15.249 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:15.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:15.530 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015287, gin 1, member 0 [ CSSD]2013-05-23 17:31:15.530 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:31:15.530 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:31:15.537 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:15.537 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:31:15.537 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:31:16.040 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:17.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:17.255 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:18.046 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:18.246 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:18.246 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:19.261 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:19.261 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:20.056 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:20.888 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:31:20.889 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:31:20.890 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:31:20.890 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108b60) proc(0x7f431c11fd30) pid(29410) version 11:1:1:4 [ CSSD]2013-05-23 17:31:20.895 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c13f730) [ CSSD]2013-05-23 17:31:20.895 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:31:20.895 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:31:20.895 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:20.895 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 547 (0x7f431bfd0190) [ CSSD]2013-05-23 17:31:20.895 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0190 (RPC#547) tag(223002d) sent to node 1 [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0223002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015288, gin 1, member 0 [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0190 (RPC#547) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0190, ret 0, client 0x7f431c13f730 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 17:31:20.896 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 547 [ CSSD]2013-05-23 17:31:20.902 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:31:20.902 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:20.902 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:20.902 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 548 (0x7f431bfd0210) [ CSSD]2013-05-23 17:31:20.902 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0210 (RPC#548) tag(224002d) sent to node 1 [ CSSD]2013-05-23 17:31:20.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0224002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:20.902 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:20.902 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0210 (RPC#548) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:20.902 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0210, ret 0, client 0x7f431c13f730 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:31:20.902 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 548 [ CSSD]2013-05-23 17:31:20.902 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:31:21.105 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:21.105 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c108b60) skgpid ospid 29410 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:21.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:21.272 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:21.495 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:31:21.496 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:31:21.497 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:31:21.497 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c11fd30) pid(29457) version 11:1:1:4 [ CSSD]2013-05-23 17:31:21.504 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:31:21.504 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:31:21.504 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0cf720 with con 0x7f431c108e60, requested num -1 [ CSSD]2013-05-23 17:31:21.504 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:31:21.504 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 549 (0x7f431bfd0290) [ CSSD]2013-05-23 17:31:21.504 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0290 (RPC#549) tag(225002d) sent to node 1 [ CSSD]2013-05-23 17:31:21.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0225002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:21.504 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015289, gin 1, member 0 [ CSSD]2013-05-23 17:31:21.504 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:31:21.505 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:31:21.505 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:31:21.505 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0290 (RPC#549) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:21.505 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0290, ret 0, client 0x7f431c0cf720 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:31:21.505 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:31:21.505 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 549 [ CSSD]2013-05-23 17:31:21.511 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:31:21.511 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:31:21.511 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13edd0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:31:21.511 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 550 (0x7f431bfd0310) [ CSSD]2013-05-23 17:31:21.511 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0310 (RPC#550) tag(226002d) sent to node 1 [ CSSD]2013-05-23 17:31:21.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0226002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:21.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:31:21.511 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0310 (RPC#550) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:21.511 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0310, ret 0, client 0x7f431c0cf720 member 0x7f431c13edd0 [ CSSD]2013-05-23 17:31:21.512 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 550 [ CSSD]2013-05-23 17:31:21.512 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c13edd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:31:21.617 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083570) allocated [ CSSD]2013-05-23 17:31:21.618 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:31:21.618 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083570 - 1,2,3 [ CSSD]2013-05-23 17:31:21.618 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108b60) proc(0x7f431c083570) pid(29464) version 11:1:1:4 [ CSSD]2013-05-23 17:31:21.632 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:31:21.633 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:31:21.636 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083570 [ CSSD]2013-05-23 17:31:21.636 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083570) con(0x7f431c108b60) skgpid ospid 29464 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:21.714 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:21.714 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c11ab50) skgpid ospid 29457 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:22.061 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:22.249 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:22.249 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:31:23.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:23.317 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:24.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:24.381 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:31:24.381 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 551 (0x7f431bfd0390) [ CSSD]2013-05-23 17:31:24.381 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0390 (RPC#551) tag(227002d) sent to node 1 [ CSSD]2013-05-23 17:31:24.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:24.381 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 551 (0x7f431bfd0390) [ CSSD]2013-05-23 17:31:24.381 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0390 (RPC#551) state 6, flags 0x202 [ CSSD]2013-05-23 17:31:24.381 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0390, ret 0, client (nil) [ CSSD]2013-05-23 17:31:24.382 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 551 [ CSSD]2013-05-23 17:31:24.983 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c1363b0) [ CSSD]2013-05-23 17:31:24.983 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c1363b0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:31:24.983 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c1363b0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:31:24.983 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 17:31:24.989 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c092510) [ CSSD]2013-05-23 17:31:24.989 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c092510), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:31:24.989 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c092510), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:31:24.989 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:31:24.989 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0920e0 [ CSSD]2013-05-23 17:31:24.989 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0920e0) con(0x7f431c133960) skgpid 24114 ospid 24114 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:25.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:25.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:26.078 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:27.251 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:27.252 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:27.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:27.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:28.132 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:29.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:29.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:30.137 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:30.792 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:30.793 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015290, gin 1, member 0 [ CSSD]2013-05-23 17:31:30.793 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:31:30.793 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:31:30.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:30.801 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:31:30.801 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:31:31.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:31.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:32.143 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:32.254 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:32.254 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:33.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:33.440 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:33.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:33.826 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015291, gin 1, member 0 [ CSSD]2013-05-23 17:31:33.826 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:31:33.826 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:31:33.832 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:33.832 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:31:33.832 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:31:34.154 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:34.461 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:31:34.461 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 552 (0x7f431bfd0410) [ CSSD]2013-05-23 17:31:34.461 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0410 (RPC#552) tag(228002d) sent to node 1 [ CSSD]2013-05-23 17:31:34.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:34.462 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 552 (0x7f431bfd0410) [ CSSD]2013-05-23 17:31:34.462 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0410 (RPC#552) state 6, flags 0x202 [ CSSD]2013-05-23 17:31:34.462 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0410, ret 0, client (nil) [ CSSD]2013-05-23 17:31:34.462 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 552 [ CSSD]2013-05-23 17:31:35.450 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:35.450 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:36.135 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 17:31:36.136 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:31:36.136 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 17:31:36.136 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c048500) pid(29594) version 11:1:1:4 [ CSSD]2013-05-23 17:31:36.141 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c048500), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:31:36.141 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:31:36.141 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:31:36.141 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:36.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 553 (0x7f431bfd0490) [ CSSD]2013-05-23 17:31:36.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0490 (RPC#553) tag(229002d) sent to node 1 [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0229002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015292, gin 1, member 0 [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0490 (RPC#553) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0490, ret 0, client 0x7f431c0cf720 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:31:36.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 553 [ CSSD]2013-05-23 17:31:36.148 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:31:36.148 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:36.148 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:36.148 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 554 (0x7f431bfd0510) [ CSSD]2013-05-23 17:31:36.148 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0510 (RPC#554) tag(22a002d) sent to node 1 [ CSSD]2013-05-23 17:31:36.149 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(022a002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:36.149 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:36.149 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0510 (RPC#554) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:36.149 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0510, ret 0, client 0x7f431c0cf720 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:36.149 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 554 [ CSSD]2013-05-23 17:31:36.149 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:31:36.178 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:36.351 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 17:31:36.351 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c108e60) skgpid ospid 29594 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:37.257 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:37.257 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:37.476 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:37.476 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:37.703 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 17:31:37.704 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:31:37.705 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 17:31:37.705 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c132e60) pid(29647) version 11:1:1:4 [ CSSD]2013-05-23 17:31:37.722 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c132e60), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:31:37.722 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:31:37.722 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:31:37.722 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:37.722 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 555 (0x7f431bfd0590) [ CSSD]2013-05-23 17:31:37.722 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0590 (RPC#555) tag(22b002d) sent to node 1 [ CSSD]2013-05-23 17:31:37.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(022b002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:37.722 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015293, gin 1, member 0 [ CSSD]2013-05-23 17:31:37.722 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:31:37.722 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:31:37.723 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:31:37.723 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0590 (RPC#555) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:37.723 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0590, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:37.723 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:31:37.723 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 555 [ CSSD]2013-05-23 17:31:37.729 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:31:37.729 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:37.729 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:37.729 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 556 (0x7f431bfd0610) [ CSSD]2013-05-23 17:31:37.729 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0610 (RPC#556) tag(22c002d) sent to node 1 [ CSSD]2013-05-23 17:31:37.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(022c002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:37.729 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:37.729 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0610 (RPC#556) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:37.729 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0610, ret 0, client 0x7f431c0e5910 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:31:37.730 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 556 [ CSSD]2013-05-23 17:31:37.730 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:31:37.932 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 17:31:37.932 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c108e60) skgpid ospid 29647 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:38.232 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:38.699 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c048500) allocated [ CSSD]2013-05-23 17:31:38.701 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:31:38.701 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c048500 - 1,2,3 [ CSSD]2013-05-23 17:31:38.701 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c048500) pid(29502/29502) version 11:1:1:4 [ CSSD]2013-05-23 17:31:38.718 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c048500), client(1/0x7f431c13f730) [ CSSD]2013-05-23 17:31:38.718 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:31:38.718 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:31:38.718 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:31:38.965 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c048500), client(2/0x7f431c0e5910) [ CSSD]2013-05-23 17:31:38.965 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0e5910) [ CSSD]2013-05-23 17:31:38.965 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:31:38.965 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 17:31:39.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:39.502 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:40.236 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:41.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:41.545 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:42.242 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:42.260 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:42.260 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:43.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:43.550 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:44.254 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:44.492 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:31:44.492 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 557 (0x7f431bfd0690) [ CSSD]2013-05-23 17:31:44.492 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0690 (RPC#557) tag(22d002d) sent to node 1 [ CSSD]2013-05-23 17:31:44.493 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:44.493 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 557 (0x7f431bfd0690) [ CSSD]2013-05-23 17:31:44.493 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0690 (RPC#557) state 6, flags 0x202 [ CSSD]2013-05-23 17:31:44.493 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0690, ret 0, client (nil) [ CSSD]2013-05-23 17:31:44.493 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 557 [ CSSD]2013-05-23 17:31:45.561 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:45.561 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:46.046 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:46.046 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015294, gin 1, member 0 [ CSSD]2013-05-23 17:31:46.046 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:31:46.046 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:31:46.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:46.052 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:31:46.052 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:31:46.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:47.262 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:47.263 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:47.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:47.567 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:48.283 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:49.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:49.640 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:50.289 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:51.265 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:51.265 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:31:51.386 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c083570) allocated [ CSSD]2013-05-23 17:31:51.387 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:31:51.387 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c083570 - 1,2,3 [ CSSD]2013-05-23 17:31:51.387 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13eeb0) proc(0x7f431c083570) pid(29746) version 11:1:1:4 [ CSSD]2013-05-23 17:31:51.393 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c083570), client(1/0x7f431c13f030) [ CSSD]2013-05-23 17:31:51.393 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:31:51.393 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f030 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:31:51.393 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:51.393 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 558 (0x7f431bfd0710) [ CSSD]2013-05-23 17:31:51.393 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0710 (RPC#558) tag(22e002d) sent to node 1 [ CSSD]2013-05-23 17:31:51.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(022e002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:51.393 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015295, gin 1, member 0 [ CSSD]2013-05-23 17:31:51.393 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090dc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:31:51.394 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:31:51.394 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:31:51.394 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0710 (RPC#558) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:51.394 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0710, ret 0, client 0x7f431c13f030 member 0x7f431c083460 [ CSSD]2013-05-23 17:31:51.394 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f030 [ CSSD]2013-05-23 17:31:51.394 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 558 [ CSSD]2013-05-23 17:31:51.400 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:31:51.400 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f030), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:51.400 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083460) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:31:51.400 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 559 (0x7f431bfd0790) [ CSSD]2013-05-23 17:31:51.401 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0790 (RPC#559) tag(22f002d) sent to node 1 [ CSSD]2013-05-23 17:31:51.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(022f002d) incarnation(174631051) [ CSSD]2013-05-23 17:31:51.401 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:31:51.401 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0790 (RPC#559) state 6, flags 0x100 [ CSSD]2013-05-23 17:31:51.401 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0790, ret 0, client 0x7f431c13f030 member 0x7f431c083460 [ CSSD]2013-05-23 17:31:51.401 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 559 [ CSSD]2013-05-23 17:31:51.401 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083460) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:31:51.604 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c083570 [ CSSD]2013-05-23 17:31:51.604 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c083570) con(0x7f431c13eeb0) skgpid ospid 29746 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:31:51.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:51.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:52.294 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:53.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:53.671 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:54.345 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:54.522 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:31:54.522 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 560 (0x7f431bfd0810) [ CSSD]2013-05-23 17:31:54.522 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0810 (RPC#560) tag(230002d) sent to node 1 [ CSSD]2013-05-23 17:31:54.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:54.523 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 560 (0x7f431bfd0810) [ CSSD]2013-05-23 17:31:54.523 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0810 (RPC#560) state 6, flags 0x202 [ CSSD]2013-05-23 17:31:54.523 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0810, ret 0, client (nil) [ CSSD]2013-05-23 17:31:54.523 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 560 [ CSSD]2013-05-23 17:31:55.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:55.683 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:56.268 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:31:56.268 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:31:56.351 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:57.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:57.689 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:31:58.363 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:31:59.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:31:59.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:00.369 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:01.271 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:01.271 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:01.295 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:01.296 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015296, gin 1, member 0 [ CSSD]2013-05-23 17:32:01.296 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:32:01.296 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:32:01.302 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:01.302 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:32:01.302 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:32:01.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:01.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:02.375 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:03.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:03.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:04.385 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:04.562 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:32:04.562 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 561 (0x7f431bfd0890) [ CSSD]2013-05-23 17:32:04.562 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0890 (RPC#561) tag(231002d) sent to node 1 [ CSSD]2013-05-23 17:32:04.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:04.563 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 561 (0x7f431bfd0890) [ CSSD]2013-05-23 17:32:04.563 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0890 (RPC#561) state 6, flags 0x202 [ CSSD]2013-05-23 17:32:04.563 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0890, ret 0, client (nil) [ CSSD]2013-05-23 17:32:04.563 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 561 [ CSSD]2013-05-23 17:32:05.782 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:05.782 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:06.274 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:06.274 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:06.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:06.635 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134680) allocated [ CSSD]2013-05-23 17:32:06.636 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:32:06.637 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134680 - 1,2,3 [ CSSD]2013-05-23 17:32:06.637 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134680) pid(29898) version 11:1:1:4 [ CSSD]2013-05-23 17:32:06.642 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134680), client(1/0x7f431c121830) [ CSSD]2013-05-23 17:32:06.642 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 17:32:06.642 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121830 with con 0x7f431c0836a0, requested num -1 [ CSSD]2013-05-23 17:32:06.642 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:06.642 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 562 (0x7f431bfd0910) [ CSSD]2013-05-23 17:32:06.643 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0910 (RPC#562) tag(232002d) sent to node 1 [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0232002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015297, gin 1, member 0 [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090ef0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0910 (RPC#562) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0910, ret 0, client 0x7f431c121830 member 0x7f431c083370 [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121830 [ CSSD]2013-05-23 17:32:06.643 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 562 [ CSSD]2013-05-23 17:32:06.649 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 17:32:06.649 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121830), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:06.649 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083370) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:06.649 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 563 (0x7f431bfd0990) [ CSSD]2013-05-23 17:32:06.649 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0990 (RPC#563) tag(233002d) sent to node 1 [ CSSD]2013-05-23 17:32:06.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0233002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:06.650 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:06.650 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0990 (RPC#563) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:06.650 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0990, ret 0, client 0x7f431c121830 member 0x7f431c083370 [ CSSD]2013-05-23 17:32:06.650 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 563 [ CSSD]2013-05-23 17:32:06.650 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083370) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:06.852 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134680 [ CSSD]2013-05-23 17:32:06.852 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134680) con(0x7f431c11ab50) skgpid ospid 29898 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:07.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:07.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:08.444 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:09.114 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 17:32:09.115 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:32:09.115 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 17:32:09.115 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22fc9c0) pid(29959) version 11:1:1:4 [ CSSD]2013-05-23 17:32:09.122 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fc9c0), client(1/0x22e01e0) [ CSSD]2013-05-23 17:32:09.122 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 17:32:09.122 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e01e0 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 17:32:09.122 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:09.122 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 564 (0x7f431bfd0a10) [ CSSD]2013-05-23 17:32:09.122 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0a10 (RPC#564) tag(234002d) sent to node 1 [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0234002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015298, gin 1, member 0 [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0a10 (RPC#564) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0a10, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e01e0 [ CSSD]2013-05-23 17:32:09.123 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 564 [ CSSD]2013-05-23 17:32:09.129 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 17:32:09.129 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e01e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:09.129 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:09.129 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 565 (0x7f431bfd0a90) [ CSSD]2013-05-23 17:32:09.129 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0a90 (RPC#565) tag(235002d) sent to node 1 [ CSSD]2013-05-23 17:32:09.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0235002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:09.129 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:09.129 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0a90 (RPC#565) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:09.129 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0a90, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 17:32:09.129 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 565 [ CSSD]2013-05-23 17:32:09.129 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:09.332 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 17:32:09.332 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x2312880) skgpid ospid 29959 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:09.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:09.841 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:10.277 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:10.277 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:32:10.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:11.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:11.846 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:12.455 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:13.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:13.851 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:14.466 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:14.588 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:32:14.589 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 566 (0x7f431bfd0b10) [ CSSD]2013-05-23 17:32:14.589 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0b10 (RPC#566) tag(236002d) sent to node 1 [ CSSD]2013-05-23 17:32:14.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:14.589 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 566 (0x7f431bfd0b10) [ CSSD]2013-05-23 17:32:14.589 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0b10 (RPC#566) state 6, flags 0x202 [ CSSD]2013-05-23 17:32:14.589 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0b10, ret 0, client (nil) [ CSSD]2013-05-23 17:32:14.589 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 566 [ CSSD]2013-05-23 17:32:15.280 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:15.280 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:15.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:15.863 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:16.472 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:16.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:16.545 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015299, gin 1, member 0 [ CSSD]2013-05-23 17:32:16.546 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:32:16.546 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:32:16.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:16.552 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:32:16.552 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:32:17.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:17.868 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:18.484 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:19.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:19.881 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:20.223 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134680) allocated [ CSSD]2013-05-23 17:32:20.224 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:32:20.225 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134680 - 1,2,3 [ CSSD]2013-05-23 17:32:20.225 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134680) pid(30257) version 11:1:1:4 [ CSSD]2013-05-23 17:32:20.283 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:20.283 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:20.356 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ad10) allocated [ CSSD]2013-05-23 17:32:20.357 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:32:20.358 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ad10 - 1,2,3 [ CSSD]2013-05-23 17:32:20.358 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090dd0) proc(0x7f431c13ad10) pid(30267) version 11:1:1:4 [ CSSD]2013-05-23 17:32:20.375 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:32:20.377 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:32:20.379 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ad10 [ CSSD]2013-05-23 17:32:20.379 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ad10) con(0x7f431c090dd0) skgpid ospid 30267 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:20.457 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(81/0x7f431c0cf720) [ CSSD]2013-05-23 17:32:20.457 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 81 (0x7f431c0cf720) [ CSSD]2013-05-23 17:32:20.457 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c0cf720 with con 0x7f431c132f30, requested num -1 [ CSSD]2013-05-23 17:32:20.457 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:32:20.457 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 567 (0x7f431bfd0b90) [ CSSD]2013-05-23 17:32:20.457 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0b90 (RPC#567) tag(237002d) sent to node 1 [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0237002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015300, gin 1, member 0 [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c090db0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0b90 (RPC#567) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0b90, ret 0, client 0x7f431c0cf720 member 0x7f431c083460 [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:32:20.458 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 567 [ CSSD]2013-05-23 17:32:20.463 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 81 (0x7f431c0cf720) [ CSSD]2013-05-23 17:32:20.463 [1191196992] >TRACE: clssgmExitGrock: client 81 (0x7f431c0cf720), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:32:20.463 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083460) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:32:20.463 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 568 (0x7f431bfd0c10) [ CSSD]2013-05-23 17:32:20.464 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0c10 (RPC#568) tag(238002d) sent to node 1 [ CSSD]2013-05-23 17:32:20.464 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0238002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:20.464 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:32:20.464 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0c10 (RPC#568) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:20.464 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0c10, ret 0, client 0x7f431c0cf720 member 0x7f431c083460 [ CSSD]2013-05-23 17:32:20.464 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 568 [ CSSD]2013-05-23 17:32:20.464 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c083460) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:20.465 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134680 [ CSSD]2013-05-23 17:32:20.465 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134680) con(0x7f431c11ab50) skgpid ospid 30257 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:20.489 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:21.886 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134680) allocated [ CSSD]2013-05-23 17:32:21.887 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:32:21.887 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:21.887 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:21.887 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134680 - 1,2,3 [ CSSD]2013-05-23 17:32:21.887 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134680) pid(30289) version 11:1:1:4 [ CSSD]2013-05-23 17:32:21.893 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134680), client(1/0x7f431c13f030) [ CSSD]2013-05-23 17:32:21.893 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:32:21.893 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f030 with con 0x7f431c132f30, requested num -1 [ CSSD]2013-05-23 17:32:21.893 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:21.893 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 569 (0x7f431bfd0c90) [ CSSD]2013-05-23 17:32:21.894 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0c90 (RPC#569) tag(239002d) sent to node 1 [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0239002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015301, gin 1, member 0 [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090dc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0c90 (RPC#569) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0c90, ret 0, client 0x7f431c13f030 member 0x7f431c083460 [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f030 [ CSSD]2013-05-23 17:32:21.894 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 569 [ CSSD]2013-05-23 17:32:21.901 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:32:21.901 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f030), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:21.901 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083460) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:21.901 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 570 (0x7f431bfd0d10) [ CSSD]2013-05-23 17:32:21.901 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0d10 (RPC#570) tag(23a002d) sent to node 1 [ CSSD]2013-05-23 17:32:21.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(023a002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:21.901 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:21.901 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0d10 (RPC#570) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:21.901 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0d10, ret 0, client 0x7f431c13f030 member 0x7f431c083460 [ CSSD]2013-05-23 17:32:21.901 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 570 [ CSSD]2013-05-23 17:32:21.901 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083460) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:22.104 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134680 [ CSSD]2013-05-23 17:32:22.104 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134680) con(0x7f431c11ab50) skgpid ospid 30289 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:22.516 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:23.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:23.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:24.567 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:24.613 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:32:24.613 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 571 (0x7f431bfd0d90) [ CSSD]2013-05-23 17:32:24.613 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0d90 (RPC#571) tag(23b002d) sent to node 1 [ CSSD]2013-05-23 17:32:24.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:24.614 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 571 (0x7f431bfd0d90) [ CSSD]2013-05-23 17:32:24.614 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0d90 (RPC#571) state 6, flags 0x202 [ CSSD]2013-05-23 17:32:24.614 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0d90, ret 0, client (nil) [ CSSD]2013-05-23 17:32:24.614 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 571 [ CSSD]2013-05-23 17:32:25.286 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:25.286 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:25.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:25.943 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:26.592 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:27.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:27.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:28.645 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:30.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:30.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:30.289 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:30.289 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:30.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:31.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:31.806 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015302, gin 1, member 0 [ CSSD]2013-05-23 17:32:31.806 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:32:31.806 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:32:31.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:31.813 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:32:31.813 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:32:32.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:32.087 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:32.656 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:34.077 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:34.077 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015303, gin 1, member 0 [ CSSD]2013-05-23 17:32:34.077 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:32:34.077 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:32:34.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:34.083 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:32:34.083 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:32:34.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:34.093 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:34.653 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:32:34.653 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 572 (0x7f431bfd0e10) [ CSSD]2013-05-23 17:32:34.654 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0e10 (RPC#572) tag(23c002d) sent to node 1 [ CSSD]2013-05-23 17:32:34.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:34.654 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 572 (0x7f431bfd0e10) [ CSSD]2013-05-23 17:32:34.654 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0e10 (RPC#572) state 6, flags 0x202 [ CSSD]2013-05-23 17:32:34.654 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd0e10, ret 0, client (nil) [ CSSD]2013-05-23 17:32:34.654 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 572 [ CSSD]2013-05-23 17:32:34.666 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:35.292 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:35.292 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:36.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:36.145 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:36.672 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:37.135 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134680) allocated [ CSSD]2013-05-23 17:32:37.135 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:32:37.136 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134680 - 1,2,3 [ CSSD]2013-05-23 17:32:37.136 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134680) pid(30436) version 11:1:1:4 [ CSSD]2013-05-23 17:32:37.142 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134680), client(1/0x7f431c13f030) [ CSSD]2013-05-23 17:32:37.142 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:32:37.142 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f030 with con 0x7f431c0836a0, requested num -1 [ CSSD]2013-05-23 17:32:37.142 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:37.142 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 573 (0x7f431bfd0e90) [ CSSD]2013-05-23 17:32:37.143 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0e90 (RPC#573) tag(23d002d) sent to node 1 [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(023d002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015304, gin 1, member 0 [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090ef0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0e90 (RPC#573) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0e90, ret 0, client 0x7f431c13f030 member 0x7f431c083370 [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f030 [ CSSD]2013-05-23 17:32:37.143 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 573 [ CSSD]2013-05-23 17:32:37.149 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:32:37.149 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f030), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:37.149 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083370) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:37.149 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 574 (0x7f431bfd0f10) [ CSSD]2013-05-23 17:32:37.149 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0f10 (RPC#574) tag(23e002d) sent to node 1 [ CSSD]2013-05-23 17:32:37.150 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(023e002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:37.150 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:37.150 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0f10 (RPC#574) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:37.150 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd0f10, ret 0, client 0x7f431c13f030 member 0x7f431c083370 [ CSSD]2013-05-23 17:32:37.150 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 574 [ CSSD]2013-05-23 17:32:37.150 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083370) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:37.352 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134680 [ CSSD]2013-05-23 17:32:37.352 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134680) con(0x7f431c11ab50) skgpid ospid 30436 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:37.976 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134680) allocated [ CSSD]2013-05-23 17:32:37.977 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:32:37.977 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134680 - 1,2,3 [ CSSD]2013-05-23 17:32:37.977 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c134680) pid(30483) version 11:1:1:4 [ CSSD]2013-05-23 17:32:37.995 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134680), client(1/0x7f431c13f030) [ CSSD]2013-05-23 17:32:37.995 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:32:37.995 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f030 with con 0x7f431c132f30, requested num -1 [ CSSD]2013-05-23 17:32:37.995 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:37.995 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 575 (0x7f431bfd0f90) [ CSSD]2013-05-23 17:32:37.995 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd0f90 (RPC#575) tag(23f002d) sent to node 1 [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(023f002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015305, gin 1, member 0 [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090dc0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd0f90 (RPC#575) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd0f90, ret 0, client 0x7f431c13f030 member 0x7f431c083460 [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f030 [ CSSD]2013-05-23 17:32:37.996 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 575 [ CSSD]2013-05-23 17:32:38.003 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f030) [ CSSD]2013-05-23 17:32:38.003 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f030), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:38.003 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083460) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:38.003 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 576 (0x7f431bfd1010) [ CSSD]2013-05-23 17:32:38.003 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1010 (RPC#576) tag(240002d) sent to node 1 [ CSSD]2013-05-23 17:32:38.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0240002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:38.004 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:38.004 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1010 (RPC#576) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:38.004 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1010, ret 0, client 0x7f431c13f030 member 0x7f431c083460 [ CSSD]2013-05-23 17:32:38.004 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 576 [ CSSD]2013-05-23 17:32:38.004 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083460) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:38.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:38.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:38.206 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134680 [ CSSD]2013-05-23 17:32:38.206 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134680) con(0x7f431c11ab50) skgpid ospid 30483 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:38.685 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:40.162 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:40.162 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:40.295 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:40.295 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:40.691 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:42.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:42.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:42.696 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:44.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:44.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:44.704 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:32:44.705 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 577 (0x7f431bfd1090) [ CSSD]2013-05-23 17:32:44.705 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1090 (RPC#577) tag(241002d) sent to node 1 [ CSSD]2013-05-23 17:32:44.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:44.705 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 577 (0x7f431bfd1090) [ CSSD]2013-05-23 17:32:44.705 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1090 (RPC#577) state 6, flags 0x202 [ CSSD]2013-05-23 17:32:44.705 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1090, ret 0, client (nil) [ CSSD]2013-05-23 17:32:44.705 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 577 [ CSSD]2013-05-23 17:32:44.747 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:45.298 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:45.298 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:46.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:46.185 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:46.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:47.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:47.066 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015306, gin 1, member 0 [ CSSD]2013-05-23 17:32:47.066 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:32:47.066 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:32:47.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:47.072 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:32:47.072 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:32:48.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:48.211 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:48.866 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:50.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:50.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:50.281 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c133050) [ CSSD]2013-05-23 17:32:50.281 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c133050), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:32:50.281 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c133050), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:32:50.281 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 17:32:50.285 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c134e00) [ CSSD]2013-05-23 17:32:50.285 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c134e00), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:32:50.285 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c134e00), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:32:50.285 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:32:50.286 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c094f50 [ CSSD]2013-05-23 17:32:50.286 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c094f50) con(0x7f431c134a80) skgpid 25120 ospid 25120 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:50.301 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:50.301 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:32:50.871 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:52.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:52.249 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:52.387 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135e90) allocated [ CSSD]2013-05-23 17:32:52.388 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:32:52.389 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135e90 - 1,2,3 [ CSSD]2013-05-23 17:32:52.389 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c132f30) proc(0x7f431c135e90) pid(30651) version 11:1:1:4 [ CSSD]2013-05-23 17:32:52.405 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c135e90), client(1/0x7f431c094e90) [ CSSD]2013-05-23 17:32:52.405 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:32:52.405 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c094e90 with con 0x7f431c121750, requested num -1 [ CSSD]2013-05-23 17:32:52.405 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:52.405 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 578 (0x7f431bfd1110) [ CSSD]2013-05-23 17:32:52.405 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1110 (RPC#578) tag(242002d) sent to node 1 [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0242002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015307, gin 1, member 0 [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1110 (RPC#578) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1110, ret 0, client 0x7f431c094e90 member 0x7f431c04b3c0 [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c094e90 [ CSSD]2013-05-23 17:32:52.406 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 578 [ CSSD]2013-05-23 17:32:52.412 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:32:52.413 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c094e90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:52.413 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c04b3c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:32:52.413 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 579 (0x7f431bfd1190) [ CSSD]2013-05-23 17:32:52.413 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1190 (RPC#579) tag(243002d) sent to node 1 [ CSSD]2013-05-23 17:32:52.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0243002d) incarnation(174631051) [ CSSD]2013-05-23 17:32:52.413 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:32:52.413 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1190 (RPC#579) state 6, flags 0x100 [ CSSD]2013-05-23 17:32:52.413 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1190, ret 0, client 0x7f431c094e90 member 0x7f431c04b3c0 [ CSSD]2013-05-23 17:32:52.413 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 579 [ CSSD]2013-05-23 17:32:52.413 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c04b3c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:32:52.616 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135e90 [ CSSD]2013-05-23 17:32:52.616 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135e90) con(0x7f431c132f30) skgpid ospid 30651 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:32:52.898 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:54.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:54.254 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:54.304 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:54.304 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:32:54.734 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:32:54.734 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 580 (0x7f431bfd1210) [ CSSD]2013-05-23 17:32:54.734 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1210 (RPC#580) tag(244002d) sent to node 1 [ CSSD]2013-05-23 17:32:54.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:54.735 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 580 (0x7f431bfd1210) [ CSSD]2013-05-23 17:32:54.735 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1210 (RPC#580) state 6, flags 0x202 [ CSSD]2013-05-23 17:32:54.735 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1210, ret 0, client (nil) [ CSSD]2013-05-23 17:32:54.735 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 580 [ CSSD]2013-05-23 17:32:54.948 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:56.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:56.273 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:56.973 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:58.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:32:58.279 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:32:58.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:32:59.306 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:32:59.306 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:00.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:00.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:00.988 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:02.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:02.299 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:02.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:02.315 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015308, gin 1, member 0 [ CSSD]2013-05-23 17:33:02.315 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:33:02.315 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:33:02.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:02.322 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:33:02.322 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:33:02.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:04.305 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:04.305 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:04.309 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:04.309 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:04.758 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:33:04.759 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 581 (0x7f431bfd1290) [ CSSD]2013-05-23 17:33:04.759 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1290 (RPC#581) tag(245002d) sent to node 1 [ CSSD]2013-05-23 17:33:04.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:04.759 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 581 (0x7f431bfd1290) [ CSSD]2013-05-23 17:33:04.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1290 (RPC#581) state 6, flags 0x202 [ CSSD]2013-05-23 17:33:04.759 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1290, ret 0, client (nil) [ CSSD]2013-05-23 17:33:04.760 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 581 [ CSSD]2013-05-23 17:33:05.038 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:06.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:06.356 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:07.044 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:07.647 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135e90) allocated [ CSSD]2013-05-23 17:33:07.647 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:33:07.648 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135e90 - 1,2,3 [ CSSD]2013-05-23 17:33:07.648 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c132f30) proc(0x7f431c135e90) pid(30811) version 11:1:1:4 [ CSSD]2013-05-23 17:33:07.664 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c135e90), client(1/0x7f431c094e90) [ CSSD]2013-05-23 17:33:07.664 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:33:07.664 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c094e90 with con 0x7f431c13ef50, requested num -1 [ CSSD]2013-05-23 17:33:07.664 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:07.665 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 582 (0x7f431bfd1310) [ CSSD]2013-05-23 17:33:07.665 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1310 (RPC#582) tag(246002d) sent to node 1 [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0246002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015309, gin 1, member 0 [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1310 (RPC#582) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1310, ret 0, client 0x7f431c094e90 member 0x7f431c04b3c0 [ CSSD]2013-05-23 17:33:07.665 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c094e90 [ CSSD]2013-05-23 17:33:07.666 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 582 [ CSSD]2013-05-23 17:33:07.672 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:33:07.672 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c094e90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:07.672 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c04b3c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:07.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 583 (0x7f431bfd1390) [ CSSD]2013-05-23 17:33:07.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1390 (RPC#583) tag(247002d) sent to node 1 [ CSSD]2013-05-23 17:33:07.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0247002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:07.672 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:07.672 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1390 (RPC#583) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:07.672 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1390, ret 0, client 0x7f431c094e90 member 0x7f431c04b3c0 [ CSSD]2013-05-23 17:33:07.672 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 583 [ CSSD]2013-05-23 17:33:07.672 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c04b3c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:33:07.875 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135e90 [ CSSD]2013-05-23 17:33:07.875 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135e90) con(0x7f431c132f30) skgpid ospid 30811 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:08.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:08.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:09.057 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:09.312 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:09.312 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:10.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:10.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:11.083 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:12.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:12.481 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:13.088 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:14.315 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:14.315 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:14.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:14.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:14.787 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:33:14.787 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 584 (0x7f431bfd1410) [ CSSD]2013-05-23 17:33:14.788 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1410 (RPC#584) tag(248002d) sent to node 1 [ CSSD]2013-05-23 17:33:14.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:14.788 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 584 (0x7f431bfd1410) [ CSSD]2013-05-23 17:33:14.788 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1410 (RPC#584) state 6, flags 0x202 [ CSSD]2013-05-23 17:33:14.788 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1410, ret 0, client (nil) [ CSSD]2013-05-23 17:33:14.788 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 584 [ CSSD]2013-05-23 17:33:15.099 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:16.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:16.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:17.105 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:17.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:17.565 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015310, gin 1, member 0 [ CSSD]2013-05-23 17:33:17.565 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:33:17.565 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:33:17.572 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:17.572 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:33:17.572 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:33:18.317 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:18.317 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:33:18.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:18.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:19.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:20.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:20.554 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:21.162 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:22.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:22.579 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:22.906 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135e90) allocated [ CSSD]2013-05-23 17:33:22.907 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:33:22.908 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135e90 - 1,2,3 [ CSSD]2013-05-23 17:33:22.908 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c132f30) proc(0x7f431c135e90) pid(30968) version 11:1:1:4 [ CSSD]2013-05-23 17:33:22.914 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c135e90), client(1/0x7f431c094e90) [ CSSD]2013-05-23 17:33:22.914 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:33:22.914 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c094e90 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:33:22.914 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:22.914 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 585 (0x7f431bfd1490) [ CSSD]2013-05-23 17:33:22.914 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1490 (RPC#585) tag(249002d) sent to node 1 [ CSSD]2013-05-23 17:33:22.914 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0249002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:22.914 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015311, gin 1, member 0 [ CSSD]2013-05-23 17:33:22.914 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:33:22.915 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:33:22.915 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:33:22.915 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1490 (RPC#585) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:22.915 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1490, ret 0, client 0x7f431c094e90 member 0x7f431c04b3c0 [ CSSD]2013-05-23 17:33:22.915 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c094e90 [ CSSD]2013-05-23 17:33:22.915 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 585 [ CSSD]2013-05-23 17:33:22.921 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:33:22.921 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c094e90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:22.921 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c04b3c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:22.921 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 586 (0x7f431bfd1510) [ CSSD]2013-05-23 17:33:22.921 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1510 (RPC#586) tag(24a002d) sent to node 1 [ CSSD]2013-05-23 17:33:22.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(024a002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:22.921 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:22.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1510 (RPC#586) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:22.921 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1510, ret 0, client 0x7f431c094e90 member 0x7f431c04b3c0 [ CSSD]2013-05-23 17:33:22.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 586 [ CSSD]2013-05-23 17:33:22.921 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c04b3c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:33:23.124 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135e90 [ CSSD]2013-05-23 17:33:23.124 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135e90) con(0x7f431c132f30) skgpid ospid 30968 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:23.255 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:23.320 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:23.320 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:24.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:24.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:24.811 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:33:24.811 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 587 (0x7f431bfd1590) [ CSSD]2013-05-23 17:33:24.811 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1590 (RPC#587) tag(24b002d) sent to node 1 [ CSSD]2013-05-23 17:33:24.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:24.812 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 587 (0x7f431bfd1590) [ CSSD]2013-05-23 17:33:24.812 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1590 (RPC#587) state 6, flags 0x202 [ CSSD]2013-05-23 17:33:24.812 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1590, ret 0, client (nil) [ CSSD]2013-05-23 17:33:24.812 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 587 [ CSSD]2013-05-23 17:33:25.280 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:26.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:26.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:27.286 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:28.323 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:28.324 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:28.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:28.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:29.168 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135e90) allocated [ CSSD]2013-05-23 17:33:29.169 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:33:29.169 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135e90 - 1,2,3 [ CSSD]2013-05-23 17:33:29.169 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c132f30) proc(0x7f431c135e90) pid(29500/29500) version 11:1:1:4 [ CSSD]2013-05-23 17:33:29.175 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c135e90), client(1/0x7f431c094e90) [ CSSD]2013-05-23 17:33:29.175 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:33:29.175 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:33:29.175 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:33:29.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:30.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:30.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:31.321 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:32.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:32.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:32.814 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:32.814 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015312, gin 1, member 0 [ CSSD]2013-05-23 17:33:32.814 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:33:32.814 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:33:32.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:32.821 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:33:32.821 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:33:33.326 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:33.326 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:33.355 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:34.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:34.344 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015313, gin 1, member 0 [ CSSD]2013-05-23 17:33:34.344 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:33:34.344 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:33:34.350 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:34.350 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:33:34.350 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:33:34.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:34.672 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:34.837 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:33:34.837 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 588 (0x7f431bfd1610) [ CSSD]2013-05-23 17:33:34.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1610 (RPC#588) tag(24c002d) sent to node 1 [ CSSD]2013-05-23 17:33:34.838 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:34.838 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 588 (0x7f431bfd1610) [ CSSD]2013-05-23 17:33:34.838 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1610 (RPC#588) state 6, flags 0x202 [ CSSD]2013-05-23 17:33:34.838 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1610, ret 0, client (nil) [ CSSD]2013-05-23 17:33:34.838 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 588 [ CSSD]2013-05-23 17:33:35.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:36.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:36.678 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:37.365 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:38.157 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129840) allocated [ CSSD]2013-05-23 17:33:38.158 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:33:38.159 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129840 - 1,2,3 [ CSSD]2013-05-23 17:33:38.159 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134df0) proc(0x7f431c129840) pid(31122) version 11:1:1:4 [ CSSD]2013-05-23 17:33:38.164 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c129840), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:33:38.164 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:33:38.164 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c083ad0, requested num -1 [ CSSD]2013-05-23 17:33:38.164 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:38.164 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 589 (0x7f431bfd1690) [ CSSD]2013-05-23 17:33:38.164 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1690 (RPC#589) tag(24d002d) sent to node 1 [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(024d002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015314, gin 1, member 0 [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c121250/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1690 (RPC#589) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1690, ret 0, client 0x7f431c0cf720 member 0x7f431c090cf0 [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:33:38.165 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 589 [ CSSD]2013-05-23 17:33:38.172 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:33:38.172 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:38.172 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c090cf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:38.172 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 590 (0x7f431bfd1710) [ CSSD]2013-05-23 17:33:38.172 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1710 (RPC#590) tag(24e002d) sent to node 1 [ CSSD]2013-05-23 17:33:38.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(024e002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:38.172 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:38.172 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1710 (RPC#590) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:38.172 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1710, ret 0, client 0x7f431c0cf720 member 0x7f431c090cf0 [ CSSD]2013-05-23 17:33:38.172 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 590 [ CSSD]2013-05-23 17:33:38.172 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c090cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:33:38.246 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092240) allocated [ CSSD]2013-05-23 17:33:38.247 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:33:38.247 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092240 - 1,2,3 [ CSSD]2013-05-23 17:33:38.247 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c092240) pid(31157) version 11:1:1:4 [ CSSD]2013-05-23 17:33:38.253 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c092240), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:33:38.253 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:33:38.253 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c04b0c0, requested num -1 [ CSSD]2013-05-23 17:33:38.253 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:38.253 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 591 (0x7f431bfd1790) [ CSSD]2013-05-23 17:33:38.254 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1790 (RPC#591) tag(24f002d) sent to node 1 [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(024f002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015315, gin 1, member 0 [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13af40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1790 (RPC#591) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1790, ret 0, client 0x7f431c0cf720 member 0x7f431c092130 [ CSSD]2013-05-23 17:33:38.254 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:33:38.255 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 591 [ CSSD]2013-05-23 17:33:38.260 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:33:38.260 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:38.260 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c092130) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:38.260 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 592 (0x7f431bfd1810) [ CSSD]2013-05-23 17:33:38.260 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1810 (RPC#592) tag(250002d) sent to node 1 [ CSSD]2013-05-23 17:33:38.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0250002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:38.260 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:38.260 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1810 (RPC#592) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:38.260 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1810, ret 0, client 0x7f431c0cf720 member 0x7f431c092130 [ CSSD]2013-05-23 17:33:38.261 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 592 [ CSSD]2013-05-23 17:33:38.261 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c092130) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:33:38.328 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:38.329 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:38.375 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129840 [ CSSD]2013-05-23 17:33:38.375 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129840) con(0x7f431c134df0) skgpid ospid 31122 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:38.463 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092240 [ CSSD]2013-05-23 17:33:38.463 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092240) con(0x7f431c13f020) skgpid ospid 31157 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:38.703 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:38.703 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:39.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:40.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:40.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:41.383 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:42.331 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:42.331 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:33:42.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:42.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:42.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:42.865 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015316, gin 1, member 0 [ CSSD]2013-05-23 17:33:42.865 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:33:42.865 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:33:42.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:42.871 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:33:42.871 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:33:42.999 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:33:42.999 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:33:43.001 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:33:43.001 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:33:43.002 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:33:43.002 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:33:43.455 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:44.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:44.793 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:44.861 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:33:44.861 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 593 (0x7f431bfd1890) [ CSSD]2013-05-23 17:33:44.862 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1890 (RPC#593) tag(251002d) sent to node 1 [ CSSD]2013-05-23 17:33:44.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:44.862 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 593 (0x7f431bfd1890) [ CSSD]2013-05-23 17:33:44.862 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1890 (RPC#593) state 6, flags 0x202 [ CSSD]2013-05-23 17:33:44.862 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1890, ret 0, client (nil) [ CSSD]2013-05-23 17:33:44.862 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 593 [ CSSD]2013-05-23 17:33:45.481 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:46.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:46.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:47.333 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:47.333 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:47.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:48.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:48.075 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015317, gin 1, member 0 [ CSSD]2013-05-23 17:33:48.075 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:33:48.075 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:33:48.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:48.081 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:33:48.081 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:33:48.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:48.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:49.498 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:50.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:50.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:51.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:52.336 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:52.336 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:52.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:52.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:53.404 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134680) allocated [ CSSD]2013-05-23 17:33:53.405 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:33:53.405 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134680 - 1,2,3 [ CSSD]2013-05-23 17:33:53.405 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134df0) proc(0x7f431c134680) pid(31287) version 11:1:1:4 [ CSSD]2013-05-23 17:33:53.410 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134680), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:33:53.410 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:33:53.410 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c1299a0, requested num -1 [ CSSD]2013-05-23 17:33:53.410 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:53.410 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 594 (0x7f431bfd1910) [ CSSD]2013-05-23 17:33:53.411 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1910 (RPC#594) tag(252002d) sent to node 1 [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0252002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015318, gin 1, member 0 [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c090cf0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1910 (RPC#594) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1910, ret 0, client 0x7f431c0cf720 member 0x7f431c121310 [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:33:53.411 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 594 [ CSSD]2013-05-23 17:33:53.417 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:33:53.417 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:53.417 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c121310) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:33:53.417 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 595 (0x7f431bfd1990) [ CSSD]2013-05-23 17:33:53.417 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1990 (RPC#595) tag(253002d) sent to node 1 [ CSSD]2013-05-23 17:33:53.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0253002d) incarnation(174631051) [ CSSD]2013-05-23 17:33:53.417 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:33:53.417 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1990 (RPC#595) state 6, flags 0x100 [ CSSD]2013-05-23 17:33:53.417 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1990, ret 0, client 0x7f431c0cf720 member 0x7f431c121310 [ CSSD]2013-05-23 17:33:53.417 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 595 [ CSSD]2013-05-23 17:33:53.417 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c121310) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:33:53.556 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:53.620 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134680 [ CSSD]2013-05-23 17:33:53.620 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134680) con(0x7f431c134df0) skgpid ospid 31287 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:54.892 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:33:54.892 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 596 (0x7f431bfd1a10) [ CSSD]2013-05-23 17:33:54.892 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1a10 (RPC#596) tag(254002d) sent to node 1 [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 596 (0x7f431bfd1a10) [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1a10 (RPC#596) state 6, flags 0x202 [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1a10, ret 0, client (nil) [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 596 [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:54.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:55.019 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e5200) [ CSSD]2013-05-23 17:33:55.019 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e5200), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:33:55.019 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e5200), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:33:55.019 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:33:55.019 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0826c0 [ CSSD]2013-05-23 17:33:55.019 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0826c0) con(0x7f431c0e6720) skgpid 25783 ospid 25783 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:55.024 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:33:55.024 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:33:55.024 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:33:55.024 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:33:55.024 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:33:55.024 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0840c0) skgpid 25785 ospid 25785 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:33:55.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:56.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:56.898 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:57.339 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:33:57.339 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:33:57.568 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:33:58.904 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:33:58.904 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:33:59.596 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:00.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:00.956 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:01.602 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:02.341 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:02.342 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:02.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:02.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:03.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:03.326 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015319, gin 1, member 0 [ CSSD]2013-05-23 17:34:03.326 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:34:03.326 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:34:03.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:03.333 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:34:03.333 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:34:03.656 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:04.917 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:34:04.917 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 597 (0x7f431bfd1a90) [ CSSD]2013-05-23 17:34:04.917 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1a90 (RPC#597) tag(255002d) sent to node 1 [ CSSD]2013-05-23 17:34:04.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:04.918 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 597 (0x7f431bfd1a90) [ CSSD]2013-05-23 17:34:04.918 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1a90 (RPC#597) state 6, flags 0x202 [ CSSD]2013-05-23 17:34:04.918 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1a90, ret 0, client (nil) [ CSSD]2013-05-23 17:34:04.918 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 597 [ CSSD]2013-05-23 17:34:04.973 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:04.973 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:05.661 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:06.344 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:06.344 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:34:06.979 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:06.979 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:07.666 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:08.663 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:34:08.664 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:34:08.664 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:34:08.664 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c10a490) pid(31445) version 11:1:1:4 [ CSSD]2013-05-23 17:34:08.670 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:34:08.670 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:34:08.670 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c134df0, requested num -1 [ CSSD]2013-05-23 17:34:08.670 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:08.670 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 598 (0x7f431bfd1b10) [ CSSD]2013-05-23 17:34:08.670 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1b10 (RPC#598) tag(256002d) sent to node 1 [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0256002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015320, gin 1, member 0 [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1b10 (RPC#598) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1b10, ret 0, client 0x7f431c12e6d0 member 0x7f431c0cf650 [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:34:08.671 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 598 [ CSSD]2013-05-23 17:34:08.677 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:34:08.677 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:08.677 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0cf650) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:08.677 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 599 (0x7f431bfd1b90) [ CSSD]2013-05-23 17:34:08.677 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1b90 (RPC#599) tag(257002d) sent to node 1 [ CSSD]2013-05-23 17:34:08.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0257002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:08.678 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:08.678 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1b90 (RPC#599) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:08.678 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1b90, ret 0, client 0x7f431c12e6d0 member 0x7f431c0cf650 [ CSSD]2013-05-23 17:34:08.678 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 599 [ CSSD]2013-05-23 17:34:08.678 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0cf650) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:34:08.880 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:34:08.880 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c0840c0) skgpid ospid 31445 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:34:08.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:08.985 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:09.679 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:11.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:11.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:11.346 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:11.346 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:11.724 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:13.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:13.032 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:13.757 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:14.340 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a490) allocated [ CSSD]2013-05-23 17:34:14.341 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:34:14.341 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a490 - 1,2,3 [ CSSD]2013-05-23 17:34:14.341 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134df0) proc(0x7f431c10a490) pid(31346/31346) version 11:1:1:4 [ CSSD]2013-05-23 17:34:14.347 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c10a490), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:34:14.347 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:34:14.347 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:34:14.347 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:34:14.941 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:34:14.941 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 600 (0x7f431bfd1c10) [ CSSD]2013-05-23 17:34:14.941 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1c10 (RPC#600) tag(258002d) sent to node 1 [ CSSD]2013-05-23 17:34:14.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:14.942 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 600 (0x7f431bfd1c10) [ CSSD]2013-05-23 17:34:14.942 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1c10 (RPC#600) state 6, flags 0x202 [ CSSD]2013-05-23 17:34:14.942 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1c10, ret 0, client (nil) [ CSSD]2013-05-23 17:34:14.942 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 600 [ CSSD]2013-05-23 17:34:15.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:15.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:15.763 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:16.349 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:16.349 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:17.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:17.070 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:17.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:18.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:18.576 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015321, gin 1, member 0 [ CSSD]2013-05-23 17:34:18.576 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:34:18.576 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:34:18.583 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:18.583 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:34:18.583 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:34:19.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:19.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:19.798 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:21.086 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:21.086 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:21.352 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:21.352 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:21.803 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:23.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:23.092 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:23.808 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:23.912 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:34:23.913 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:34:23.913 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:34:23.913 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c13ef60) pid(31598) version 11:1:1:4 [ CSSD]2013-05-23 17:34:23.918 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ef60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:34:23.918 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:34:23.918 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:34:23.918 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:23.918 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 601 (0x7f431bfd1c90) [ CSSD]2013-05-23 17:34:23.918 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1c90 (RPC#601) tag(259002d) sent to node 1 [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0259002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015322, gin 1, member 0 [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1c90 (RPC#601) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1c90, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:34:23.919 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 601 [ CSSD]2013-05-23 17:34:23.925 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:34:23.925 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:23.925 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095020) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:23.925 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 602 (0x7f431bfd1d10) [ CSSD]2013-05-23 17:34:23.925 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1d10 (RPC#602) tag(25a002d) sent to node 1 [ CSSD]2013-05-23 17:34:23.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(025a002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:23.926 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:23.926 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1d10 (RPC#602) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:23.926 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1d10, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:34:23.926 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 602 [ CSSD]2013-05-23 17:34:23.926 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095020) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:34:24.128 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:34:24.128 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c0e5200) skgpid ospid 31598 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:34:24.968 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:34:24.969 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 603 (0x7f431bfd1d90) [ CSSD]2013-05-23 17:34:24.969 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1d90 (RPC#603) tag(25b002d) sent to node 1 [ CSSD]2013-05-23 17:34:24.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:24.969 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 603 (0x7f431bfd1d90) [ CSSD]2013-05-23 17:34:24.969 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1d90 (RPC#603) state 6, flags 0x202 [ CSSD]2013-05-23 17:34:24.969 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1d90, ret 0, client (nil) [ CSSD]2013-05-23 17:34:24.969 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 603 [ CSSD]2013-05-23 17:34:25.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:25.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:25.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:26.355 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:26.355 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:27.111 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:27.111 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:27.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:29.136 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:29.136 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:29.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:30.156 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:34:30.157 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:34:30.158 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:34:30.158 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:34:30.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:30.242 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015323, gin 1, member 0 [ CSSD]2013-05-23 17:34:30.242 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:34:30.242 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:34:30.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:30.248 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:34:30.248 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:34:31.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:31.167 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:31.358 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:31.358 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:31.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:33.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:33.174 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:33.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:33.830 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015324, gin 1, member 0 [ CSSD]2013-05-23 17:34:33.830 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:34:33.830 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:34:33.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:33.837 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:34:33.837 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:34:33.907 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:34.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:34.610 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015325, gin 1, member 0 [ CSSD]2013-05-23 17:34:34.610 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:34:34.610 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:34:34.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:34.617 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:34:34.617 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:34:34.993 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:34:34.994 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 604 (0x7f431bfd1e10) [ CSSD]2013-05-23 17:34:34.994 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1e10 (RPC#604) tag(25c002d) sent to node 1 [ CSSD]2013-05-23 17:34:34.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:34.994 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 604 (0x7f431bfd1e10) [ CSSD]2013-05-23 17:34:34.994 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1e10 (RPC#604) state 6, flags 0x202 [ CSSD]2013-05-23 17:34:34.994 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd1e10, ret 0, client (nil) [ CSSD]2013-05-23 17:34:34.995 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 604 [ CSSD]2013-05-23 17:34:35.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:35.185 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:35.912 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:36.360 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:36.360 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:37.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:37.192 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:37.937 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:38.499 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:34:38.500 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:34:38.501 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:34:38.501 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135110) proc(0x7f431c082370) pid(31736) version 11:1:1:4 [ CSSD]2013-05-23 17:34:38.507 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:34:38.507 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:34:38.507 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0951a0, requested num -1 [ CSSD]2013-05-23 17:34:38.507 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:38.507 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 605 (0x7f431bfd1e90) [ CSSD]2013-05-23 17:34:38.508 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1e90 (RPC#605) tag(25d002d) sent to node 1 [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(025d002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015326, gin 1, member 0 [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1e90 (RPC#605) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1e90, ret 0, client 0x7f431c0cf720 member 0x7f431c083a00 [ CSSD]2013-05-23 17:34:38.508 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:34:38.509 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 605 [ CSSD]2013-05-23 17:34:38.514 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:34:38.514 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:38.514 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:38.514 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 606 (0x7f431bfd1f10) [ CSSD]2013-05-23 17:34:38.514 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1f10 (RPC#606) tag(25e002d) sent to node 1 [ CSSD]2013-05-23 17:34:38.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(025e002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:38.515 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:38.515 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1f10 (RPC#606) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:38.515 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd1f10, ret 0, client 0x7f431c0cf720 member 0x7f431c083a00 [ CSSD]2013-05-23 17:34:38.515 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 606 [ CSSD]2013-05-23 17:34:38.515 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:34:38.717 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:34:38.717 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c135110) skgpid ospid 31736 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:34:39.159 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:34:39.160 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:34:39.160 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:34:39.160 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(31751) version 11:1:1:4 [ CSSD]2013-05-23 17:34:39.166 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:34:39.166 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:34:39.166 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:34:39.166 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:39.166 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 607 (0x7f431bfd1f90) [ CSSD]2013-05-23 17:34:39.167 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd1f90 (RPC#607) tag(25f002d) sent to node 1 [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(025f002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015327, gin 1, member 0 [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd1f90 (RPC#607) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd1f90, ret 0, client 0x7f431c135120 member 0x7f431c083930 [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:34:39.167 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 607 [ CSSD]2013-05-23 17:34:39.173 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:34:39.173 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:39.173 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:39.173 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 608 (0x7f431bfd2010) [ CSSD]2013-05-23 17:34:39.173 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2010 (RPC#608) tag(260002d) sent to node 1 [ CSSD]2013-05-23 17:34:39.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0260002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:39.173 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:39.173 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2010 (RPC#608) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:39.173 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2010, ret 0, client 0x7f431c135120 member 0x7f431c083930 [ CSSD]2013-05-23 17:34:39.173 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 608 [ CSSD]2013-05-23 17:34:39.173 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:34:39.217 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:39.217 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:39.376 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:34:39.376 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 31751 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:34:39.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:41.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:41.268 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:41.363 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:41.363 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:41.995 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:43.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:43.274 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:44.089 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:45.017 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:34:45.017 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 609 (0x7f431bfd2090) [ CSSD]2013-05-23 17:34:45.017 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2090 (RPC#609) tag(261002d) sent to node 1 [ CSSD]2013-05-23 17:34:45.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:45.018 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 609 (0x7f431bfd2090) [ CSSD]2013-05-23 17:34:45.018 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2090 (RPC#609) state 6, flags 0x202 [ CSSD]2013-05-23 17:34:45.018 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2090, ret 0, client (nil) [ CSSD]2013-05-23 17:34:45.018 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 609 [ CSSD]2013-05-23 17:34:45.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:45.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:46.095 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:46.367 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:46.367 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:47.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:47.292 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:48.101 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:49.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:49.090 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015328, gin 1, member 0 [ CSSD]2013-05-23 17:34:49.090 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:34:49.090 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:34:49.096 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:49.096 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:34:49.096 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:34:49.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:49.317 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:50.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:51.370 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:51.370 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:34:51.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:51.409 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:52.138 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:53.414 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:53.414 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:54.169 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:54.408 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:34:54.409 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:34:54.410 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:34:54.410 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c13ef60) pid(31918) version 11:1:1:4 [ CSSD]2013-05-23 17:34:54.416 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ef60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:34:54.416 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:34:54.416 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0951a0, requested num -1 [ CSSD]2013-05-23 17:34:54.416 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:54.416 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 610 (0x7f431bfd2110) [ CSSD]2013-05-23 17:34:54.416 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2110 (RPC#610) tag(262002d) sent to node 1 [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0262002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015329, gin 1, member 0 [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2110 (RPC#610) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2110, ret 0, client 0x7f431c135120 member 0x7f431c083930 [ CSSD]2013-05-23 17:34:54.417 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:34:54.418 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 610 [ CSSD]2013-05-23 17:34:54.423 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:34:54.423 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:54.423 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:34:54.424 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 611 (0x7f431bfd2190) [ CSSD]2013-05-23 17:34:54.424 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2190 (RPC#611) tag(263002d) sent to node 1 [ CSSD]2013-05-23 17:34:54.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0263002d) incarnation(174631051) [ CSSD]2013-05-23 17:34:54.424 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:34:54.424 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2190 (RPC#611) state 6, flags 0x100 [ CSSD]2013-05-23 17:34:54.424 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2190, ret 0, client 0x7f431c135120 member 0x7f431c083930 [ CSSD]2013-05-23 17:34:54.424 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 611 [ CSSD]2013-05-23 17:34:54.424 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:34:54.627 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:34:54.627 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c0cf710) skgpid ospid 31918 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:34:55.043 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:34:55.043 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 612 (0x7f431bfd2210) [ CSSD]2013-05-23 17:34:55.043 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2210 (RPC#612) tag(264002d) sent to node 1 [ CSSD]2013-05-23 17:34:55.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:55.044 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 612 (0x7f431bfd2210) [ CSSD]2013-05-23 17:34:55.044 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2210 (RPC#612) state 6, flags 0x202 [ CSSD]2013-05-23 17:34:55.044 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2210, ret 0, client (nil) [ CSSD]2013-05-23 17:34:55.044 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 612 [ CSSD]2013-05-23 17:34:55.373 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:34:55.373 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:34:55.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:55.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:56.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:57.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:57.471 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:34:58.187 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:34:59.477 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:34:59.477 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:00.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:00.376 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:00.376 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:01.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:01.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:02.218 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:03.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:03.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:04.270 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:04.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:04.339 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015330, gin 1, member 0 [ CSSD]2013-05-23 17:35:04.339 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:35:04.339 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:35:04.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:04.344 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:35:04.345 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:35:05.068 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:35:05.069 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 613 (0x7f431bfd2290) [ CSSD]2013-05-23 17:35:05.069 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2290 (RPC#613) tag(265002d) sent to node 1 [ CSSD]2013-05-23 17:35:05.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:05.069 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 613 (0x7f431bfd2290) [ CSSD]2013-05-23 17:35:05.069 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2290 (RPC#613) state 6, flags 0x202 [ CSSD]2013-05-23 17:35:05.069 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2290, ret 0, client (nil) [ CSSD]2013-05-23 17:35:05.070 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 613 [ CSSD]2013-05-23 17:35:05.379 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:05.379 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:05.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:05.586 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:06.276 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:07.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:07.592 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:08.287 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:08.395 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:35:08.395 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:35:08.396 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:35:08.396 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135110) proc(0x7f431c082370) pid(32076) version 11:1:1:4 [ CSSD]2013-05-23 17:35:08.982 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:35:08.982 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c135110) skgpid ospid 32076 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:35:09.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:09.597 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:09.658 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:35:09.659 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:35:09.659 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:35:09.659 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(32177) version 11:1:1:4 [ CSSD]2013-05-23 17:35:09.665 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:35:09.665 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:09.665 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:35:09.665 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:09.665 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 614 (0x7f431bfd2310) [ CSSD]2013-05-23 17:35:09.665 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2310 (RPC#614) tag(266002d) sent to node 1 [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0266002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015331, gin 1, member 0 [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2310 (RPC#614) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2310, ret 0, client 0x7f431c135120 member 0x7f431c083930 [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:35:09.666 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 614 [ CSSD]2013-05-23 17:35:09.672 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:09.672 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:09.672 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:09.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 615 (0x7f431bfd2390) [ CSSD]2013-05-23 17:35:09.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2390 (RPC#615) tag(267002d) sent to node 1 [ CSSD]2013-05-23 17:35:09.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0267002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:09.672 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:09.672 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2390 (RPC#615) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:09.672 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2390, ret 0, client 0x7f431c135120 member 0x7f431c083930 [ CSSD]2013-05-23 17:35:09.673 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 615 [ CSSD]2013-05-23 17:35:09.673 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:35:09.875 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:35:09.875 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 32177 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:35:10.293 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:10.383 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:10.383 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:11.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:11.609 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:12.300 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:13.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:13.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:14.310 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:14.385 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:14.385 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:35:15.094 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:35:15.094 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 616 (0x7f431bfd2410) [ CSSD]2013-05-23 17:35:15.094 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2410 (RPC#616) tag(268002d) sent to node 1 [ CSSD]2013-05-23 17:35:15.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:15.095 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 616 (0x7f431bfd2410) [ CSSD]2013-05-23 17:35:15.095 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2410 (RPC#616) state 6, flags 0x202 [ CSSD]2013-05-23 17:35:15.095 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2410, ret 0, client (nil) [ CSSD]2013-05-23 17:35:15.095 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 616 [ CSSD]2013-05-23 17:35:15.667 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:15.667 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:16.335 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:17.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:17.674 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:18.368 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:19.388 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:19.388 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:19.588 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:19.588 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015332, gin 1, member 0 [ CSSD]2013-05-23 17:35:19.588 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:35:19.588 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:35:19.596 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:19.596 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:35:19.596 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:35:19.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:19.685 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:20.373 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:21.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:21.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:22.379 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:23.716 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:23.716 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:24.389 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:24.391 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:24.391 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:24.907 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:35:24.908 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:35:24.908 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:35:24.908 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c13ef60) pid(32332) version 11:1:1:4 [ CSSD]2013-05-23 17:35:24.914 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ef60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:35:24.914 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:24.914 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:35:24.914 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:24.914 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 617 (0x7f431bfd2490) [ CSSD]2013-05-23 17:35:24.914 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2490 (RPC#617) tag(269002d) sent to node 1 [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0269002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015333, gin 1, member 0 [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2490 (RPC#617) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2490, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:35:24.915 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 617 [ CSSD]2013-05-23 17:35:24.921 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:24.921 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:24.921 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095020) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:24.921 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 618 (0x7f431bfd2510) [ CSSD]2013-05-23 17:35:24.921 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2510 (RPC#618) tag(26a002d) sent to node 1 [ CSSD]2013-05-23 17:35:24.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(026a002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:24.921 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:24.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2510 (RPC#618) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:24.921 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2510, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:35:24.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 618 [ CSSD]2013-05-23 17:35:24.921 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095020) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:35:25.121 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:35:25.121 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 619 (0x7f431bfd2590) [ CSSD]2013-05-23 17:35:25.121 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2590 (RPC#619) tag(26b002d) sent to node 1 [ CSSD]2013-05-23 17:35:25.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:25.122 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 619 (0x7f431bfd2590) [ CSSD]2013-05-23 17:35:25.122 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2590 (RPC#619) state 6, flags 0x202 [ CSSD]2013-05-23 17:35:25.122 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2590, ret 0, client (nil) [ CSSD]2013-05-23 17:35:25.122 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 619 [ CSSD]2013-05-23 17:35:25.123 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:35:25.123 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c0e5200) skgpid ospid 32332 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:35:25.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:25.768 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:26.434 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:27.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:27.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:28.468 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:29.394 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:29.394 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:29.785 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:29.785 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:30.474 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:31.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:31.791 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:32.480 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:33.397 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:33.397 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:35:33.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:33.816 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:34.491 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:34.841 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:34.841 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015334, gin 1, member 0 [ CSSD]2013-05-23 17:35:34.841 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:35:34.841 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:35:34.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:34.848 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:35:34.848 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:35:34.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:34.869 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015335, gin 1, member 0 [ CSSD]2013-05-23 17:35:34.869 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:35:34.869 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:35:34.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:34.875 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:35:34.875 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:35:35.145 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:35:35.145 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 620 (0x7f431bfd2610) [ CSSD]2013-05-23 17:35:35.145 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2610 (RPC#620) tag(26c002d) sent to node 1 [ CSSD]2013-05-23 17:35:35.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:35.146 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 620 (0x7f431bfd2610) [ CSSD]2013-05-23 17:35:35.146 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2610 (RPC#620) state 6, flags 0x202 [ CSSD]2013-05-23 17:35:35.146 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2610, ret 0, client (nil) [ CSSD]2013-05-23 17:35:35.146 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 620 [ CSSD]2013-05-23 17:35:35.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:35.817 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:36.516 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:37.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:37.823 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:38.399 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:38.400 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:38.588 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:38.765 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:35:38.766 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:35:38.766 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:35:38.766 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c082370) pid(32473) version 11:1:1:4 [ CSSD]2013-05-23 17:35:38.772 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:35:38.772 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:38.772 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0951a0, requested num -1 [ CSSD]2013-05-23 17:35:38.772 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:38.772 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 621 (0x7f431bfd2690) [ CSSD]2013-05-23 17:35:38.772 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2690 (RPC#621) tag(26d002d) sent to node 1 [ CSSD]2013-05-23 17:35:38.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(026d002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:38.772 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015336, gin 1, member 0 [ CSSD]2013-05-23 17:35:38.772 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:35:38.773 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:35:38.773 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:35:38.773 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2690 (RPC#621) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:38.773 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2690, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:35:38.773 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:35:38.773 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 621 [ CSSD]2013-05-23 17:35:38.779 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:38.779 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:38.779 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:38.779 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 622 (0x7f431bfd2710) [ CSSD]2013-05-23 17:35:38.779 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2710 (RPC#622) tag(26e002d) sent to node 1 [ CSSD]2013-05-23 17:35:38.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(026e002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:38.779 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:38.779 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2710 (RPC#622) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:38.779 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2710, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:35:38.779 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 622 [ CSSD]2013-05-23 17:35:38.779 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:35:38.981 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:35:38.981 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0840c0) skgpid ospid 32473 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:35:39.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:39.876 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:40.163 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:35:40.164 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:35:40.164 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:35:40.164 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c082370) pid(32509) version 11:1:1:4 [ CSSD]2013-05-23 17:35:40.169 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:35:40.170 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:35:40.170 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:35:40.170 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:40.170 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 623 (0x7f431bfd2790) [ CSSD]2013-05-23 17:35:40.170 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2790 (RPC#623) tag(26f002d) sent to node 1 [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(026f002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015337, gin 1, member 0 [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2790 (RPC#623) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2790, ret 0, client 0x7f431c0cf720 member 0x7f431c083930 [ CSSD]2013-05-23 17:35:40.170 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:35:40.171 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 623 [ CSSD]2013-05-23 17:35:40.176 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:35:40.176 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:40.176 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083930) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:40.176 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 624 (0x7f431bfd2810) [ CSSD]2013-05-23 17:35:40.177 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2810 (RPC#624) tag(270002d) sent to node 1 [ CSSD]2013-05-23 17:35:40.177 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0270002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:40.177 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:40.177 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2810 (RPC#624) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:40.177 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2810, ret 0, client 0x7f431c0cf720 member 0x7f431c083930 [ CSSD]2013-05-23 17:35:40.177 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 624 [ CSSD]2013-05-23 17:35:40.177 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083930) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:35:40.380 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:35:40.380 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0840c0) skgpid ospid 32509 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:35:40.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:41.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:41.882 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:42.600 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:43.402 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:43.402 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:43.888 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:43.888 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:44.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:45.173 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:35:45.173 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 625 (0x7f431bfd2890) [ CSSD]2013-05-23 17:35:45.173 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2890 (RPC#625) tag(271002d) sent to node 1 [ CSSD]2013-05-23 17:35:45.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:45.173 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 625 (0x7f431bfd2890) [ CSSD]2013-05-23 17:35:45.173 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2890 (RPC#625) state 6, flags 0x202 [ CSSD]2013-05-23 17:35:45.174 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2890, ret 0, client (nil) [ CSSD]2013-05-23 17:35:45.174 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 625 [ CSSD]2013-05-23 17:35:45.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:45.917 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:46.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:47.923 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:47.923 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:48.405 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:48.405 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:48.619 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:49.976 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:49.976 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:50.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:50.088 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015338, gin 1, member 0 [ CSSD]2013-05-23 17:35:50.088 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:35:50.088 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:35:50.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:50.094 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:35:50.094 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:35:50.625 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:51.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:51.983 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:52.670 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:53.408 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:53.409 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:35:53.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:53.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:54.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:55.204 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:35:55.204 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 626 (0x7f431bfd2910) [ CSSD]2013-05-23 17:35:55.204 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2910 (RPC#626) tag(272002d) sent to node 1 [ CSSD]2013-05-23 17:35:55.205 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:55.205 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 626 (0x7f431bfd2910) [ CSSD]2013-05-23 17:35:55.205 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2910 (RPC#626) state 6, flags 0x202 [ CSSD]2013-05-23 17:35:55.205 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2910, ret 0, client (nil) [ CSSD]2013-05-23 17:35:55.205 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 626 [ CSSD]2013-05-23 17:35:55.412 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:35:55.413 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:35:55.414 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:35:55.414 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c13ef60) pid(32653) version 11:1:1:4 [ CSSD]2013-05-23 17:35:55.419 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ef60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:35:55.419 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:55.419 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:35:55.419 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:55.419 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 627 (0x7f431bfd2990) [ CSSD]2013-05-23 17:35:55.420 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2990 (RPC#627) tag(273002d) sent to node 1 [ CSSD]2013-05-23 17:35:55.420 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0273002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:55.420 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015339, gin 1, member 0 [ CSSD]2013-05-23 17:35:55.420 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c082440/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:35:55.421 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:35:55.421 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:35:55.421 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2990 (RPC#627) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:55.421 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2990, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:35:55.421 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:35:55.421 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 627 [ CSSD]2013-05-23 17:35:55.427 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:35:55.427 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:55.427 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095020) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:35:55.427 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 628 (0x7f431bfd2a10) [ CSSD]2013-05-23 17:35:55.427 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2a10 (RPC#628) tag(274002d) sent to node 1 [ CSSD]2013-05-23 17:35:55.428 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0274002d) incarnation(174631051) [ CSSD]2013-05-23 17:35:55.428 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:35:55.428 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2a10 (RPC#628) state 6, flags 0x100 [ CSSD]2013-05-23 17:35:55.428 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2a10, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:35:55.428 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 628 [ CSSD]2013-05-23 17:35:55.428 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095020) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:35:55.630 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:35:55.630 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c0e5200) skgpid ospid 32653 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:35:55.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:55.998 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:56.707 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:35:57.411 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:35:57.411 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:35:58.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:35:58.023 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:35:58.719 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:00.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:00.097 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:00.725 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:02.123 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:02.123 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:02.414 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:02.414 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:02.730 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:04.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:04.129 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:04.783 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:05.231 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:36:05.232 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 629 (0x7f431bfd2a90) [ CSSD]2013-05-23 17:36:05.232 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2a90 (RPC#629) tag(275002d) sent to node 1 [ CSSD]2013-05-23 17:36:05.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:05.232 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 629 (0x7f431bfd2a90) [ CSSD]2013-05-23 17:36:05.232 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2a90 (RPC#629) state 6, flags 0x202 [ CSSD]2013-05-23 17:36:05.232 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2a90, ret 0, client (nil) [ CSSD]2013-05-23 17:36:05.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 629 [ CSSD]2013-05-23 17:36:05.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:05.348 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015340, gin 1, member 0 [ CSSD]2013-05-23 17:36:05.348 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:36:05.348 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:36:05.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:05.355 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:36:05.355 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:36:06.180 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:06.180 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:06.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:07.417 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:07.417 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:08.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:08.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:08.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:10.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:10.197 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:10.663 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:36:10.664 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:36:10.664 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:36:10.664 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c13ef60) pid(327) version 11:1:1:4 [ CSSD]2013-05-23 17:36:10.670 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ef60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:36:10.670 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:36:10.670 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:36:10.670 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:10.670 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 630 (0x7f431bfd2b10) [ CSSD]2013-05-23 17:36:10.671 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2b10 (RPC#630) tag(276002d) sent to node 1 [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0276002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015341, gin 1, member 0 [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2b10 (RPC#630) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2b10, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:36:10.671 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 630 [ CSSD]2013-05-23 17:36:10.677 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:36:10.677 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:10.677 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095020) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:10.677 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 631 (0x7f431bfd2b90) [ CSSD]2013-05-23 17:36:10.677 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2b90 (RPC#631) tag(277002d) sent to node 1 [ CSSD]2013-05-23 17:36:10.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0277002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:10.677 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:10.677 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2b90 (RPC#631) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:10.677 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2b90, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:36:10.677 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 631 [ CSSD]2013-05-23 17:36:10.677 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c095020) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:36:10.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:10.880 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:36:10.880 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c0840c0) skgpid ospid 327 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:12.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:12.223 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:12.420 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:12.420 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:12.830 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:14.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:14.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:14.901 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:15.368 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:36:15.368 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 632 (0x7f431bfd2c10) [ CSSD]2013-05-23 17:36:15.368 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2c10 (RPC#632) tag(278002d) sent to node 1 [ CSSD]2013-05-23 17:36:15.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:15.369 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 632 (0x7f431bfd2c10) [ CSSD]2013-05-23 17:36:15.369 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2c10 (RPC#632) state 6, flags 0x202 [ CSSD]2013-05-23 17:36:15.369 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2c10, ret 0, client (nil) [ CSSD]2013-05-23 17:36:15.369 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 632 [ CSSD]2013-05-23 17:36:16.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:16.280 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:16.907 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:17.422 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:17.423 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:18.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:18.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:18.920 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:20.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:20.297 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:20.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:20.607 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015342, gin 1, member 0 [ CSSD]2013-05-23 17:36:20.608 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:36:20.608 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:36:20.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:20.615 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:36:20.615 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:36:20.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:21.792 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ef60) allocated [ CSSD]2013-05-23 17:36:21.792 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:36:21.793 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ef60 - 1,2,3 [ CSSD]2013-05-23 17:36:21.793 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c13ef60) pid(452) version 11:1:1:4 [ CSSD]2013-05-23 17:36:21.799 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13ef60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:36:21.799 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:36:21.799 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:36:21.799 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:36:21.799 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 633 (0x7f431bfd2c90) [ CSSD]2013-05-23 17:36:21.799 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2c90 (RPC#633) tag(279002d) sent to node 1 [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0279002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015343, gin 1, member 0 [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083630/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2c90 (RPC#633) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2c90, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:36:21.800 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 633 [ CSSD]2013-05-23 17:36:21.806 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:36:21.806 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:36:21.806 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c095020) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:36:21.806 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 634 (0x7f431bfd2d10) [ CSSD]2013-05-23 17:36:21.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2d10 (RPC#634) tag(27a002d) sent to node 1 [ CSSD]2013-05-23 17:36:21.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(027a002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:21.807 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:36:21.807 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2d10 (RPC#634) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:21.807 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2d10, ret 0, client 0x7f431c135120 member 0x7f431c095020 [ CSSD]2013-05-23 17:36:21.807 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 634 [ CSSD]2013-05-23 17:36:21.807 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c095020) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:36:21.918 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0922e0) allocated [ CSSD]2013-05-23 17:36:21.918 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:36:21.919 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0922e0 - 1,2,3 [ CSSD]2013-05-23 17:36:21.919 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c0922e0) pid(457) version 11:1:1:4 [ CSSD]2013-05-23 17:36:21.945 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:36:21.946 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:36:21.949 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0922e0 [ CSSD]2013-05-23 17:36:21.949 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0922e0) con(0x7f431c0840c0) skgpid ospid 457 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:22.009 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ef60 [ CSSD]2013-05-23 17:36:22.009 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ef60) con(0x7f431c0e5200) skgpid ospid 452 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:22.303 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:22.303 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:22.425 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:22.425 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:22.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:24.309 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:24.309 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:24.983 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:25.061 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:25.061 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:36:25.061 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c109f80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:36:25.061 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:36:25.061 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b850 [ CSSD]2013-05-23 17:36:25.061 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b850) con(0x7f431c0e6e20) skgpid 27624 ospid 27624 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:25.393 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:36:25.393 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 635 (0x7f431bfd2d90) [ CSSD]2013-05-23 17:36:25.393 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2d90 (RPC#635) tag(27b002d) sent to node 1 [ CSSD]2013-05-23 17:36:25.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:25.394 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 635 (0x7f431bfd2d90) [ CSSD]2013-05-23 17:36:25.394 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2d90 (RPC#635) state 6, flags 0x202 [ CSSD]2013-05-23 17:36:25.394 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2d90, ret 0, client (nil) [ CSSD]2013-05-23 17:36:25.394 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 635 [ CSSD]2013-05-23 17:36:25.923 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:36:25.924 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:36:25.924 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:36:25.924 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c10a190) pid(507) version 11:1:1:4 [ CSSD]2013-05-23 17:36:25.930 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c10a190), client(1/0x7f431c092940) [ CSSD]2013-05-23 17:36:25.930 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c092940) [ CSSD]2013-05-23 17:36:25.930 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c092940 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:36:25.930 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:25.930 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 636 (0x7f431bfd2e10) [ CSSD]2013-05-23 17:36:25.930 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2e10 (RPC#636) tag(27c002d) sent to node 1 [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(027c002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015344, gin 1, member 0 [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c04b0c0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2e10 (RPC#636) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2e10, ret 0, client 0x7f431c092940 member 0x7f431c13f950 [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c092940 [ CSSD]2013-05-23 17:36:25.931 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 636 [ CSSD]2013-05-23 17:36:25.936 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c092940) [ CSSD]2013-05-23 17:36:25.936 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c092940), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:25.936 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13f950) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:25.936 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 637 (0x7f431bfd2e90) [ CSSD]2013-05-23 17:36:25.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2e90 (RPC#637) tag(27d002d) sent to node 1 [ CSSD]2013-05-23 17:36:25.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(027d002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:25.937 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:25.937 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2e90 (RPC#637) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:25.937 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd2e90, ret 0, client 0x7f431c092940 member 0x7f431c13f950 [ CSSD]2013-05-23 17:36:25.937 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 637 [ CSSD]2013-05-23 17:36:25.937 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13f950) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:36:26.139 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:36:26.139 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c12f0c0) skgpid ospid 507 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:26.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:26.320 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:27.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:27.166 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:36:27.166 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:36:27.167 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:36:27.167 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6e20) proc(0x7f431c10a190) pid(482/482) version 11:1:1:4 [ CSSD]2013-05-23 17:36:27.172 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c10a190), client(1/0x7f431c092940) [ CSSD]2013-05-23 17:36:27.172 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c092940) [ CSSD]2013-05-23 17:36:27.172 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:36:27.172 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:36:27.428 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:27.428 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:28.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:28.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:29.020 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:30.378 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:30.378 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:31.025 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:32.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:32.385 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:32.431 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:32.431 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:33.031 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:34.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:34.411 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:35.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:35.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:35.137 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015345, gin 1, member 0 [ CSSD]2013-05-23 17:36:35.137 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:36:35.137 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:36:35.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:35.145 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:36:35.145 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:36:35.420 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:36:35.420 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 638 (0x7f431bfd2f10) [ CSSD]2013-05-23 17:36:35.421 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2f10 (RPC#638) tag(27e002d) sent to node 1 [ CSSD]2013-05-23 17:36:35.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:35.421 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 638 (0x7f431bfd2f10) [ CSSD]2013-05-23 17:36:35.421 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2f10 (RPC#638) state 6, flags 0x202 [ CSSD]2013-05-23 17:36:35.421 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd2f10, ret 0, client (nil) [ CSSD]2013-05-23 17:36:35.421 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 638 [ CSSD]2013-05-23 17:36:35.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:35.858 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015346, gin 1, member 0 [ CSSD]2013-05-23 17:36:35.858 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:36:35.858 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:36:35.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:35.865 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:36:35.865 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:36:36.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:36.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:37.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:37.433 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:37.434 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:38.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:38.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:39.023 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:36:39.024 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:36:39.024 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:36:39.024 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(654) version 11:1:1:4 [ CSSD]2013-05-23 17:36:39.030 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:36:39.030 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:39.030 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:36:39.030 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:39.030 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 639 (0x7f431bfd2f90) [ CSSD]2013-05-23 17:36:39.030 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd2f90 (RPC#639) tag(27f002d) sent to node 1 [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(027f002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015347, gin 1, member 0 [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd2f90 (RPC#639) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd2f90, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:36:39.031 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 639 [ CSSD]2013-05-23 17:36:39.038 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:39.038 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:39.038 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:39.038 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 640 (0x7f431bfd3010) [ CSSD]2013-05-23 17:36:39.038 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3010 (RPC#640) tag(280002d) sent to node 1 [ CSSD]2013-05-23 17:36:39.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0280002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:39.038 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:39.038 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3010 (RPC#640) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:39.038 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3010, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:36:39.039 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 640 [ CSSD]2013-05-23 17:36:39.039 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:36:39.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:39.241 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:36:39.241 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 654 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:40.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:40.479 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:41.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:41.172 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:36:41.174 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:36:41.174 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:36:41.174 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c082370) pid(676) version 11:1:1:4 [ CSSD]2013-05-23 17:36:41.180 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:36:41.180 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:41.180 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:36:41.180 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:41.180 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 641 (0x7f431bfd3090) [ CSSD]2013-05-23 17:36:41.180 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3090 (RPC#641) tag(281002d) sent to node 1 [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0281002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015348, gin 1, member 0 [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3090 (RPC#641) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3090, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:36:41.181 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 641 [ CSSD]2013-05-23 17:36:41.188 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:41.188 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:41.188 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:41.188 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 642 (0x7f431bfd3110) [ CSSD]2013-05-23 17:36:41.188 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3110 (RPC#642) tag(282002d) sent to node 1 [ CSSD]2013-05-23 17:36:41.188 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0282002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:41.188 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:41.188 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3110 (RPC#642) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:41.188 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3110, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:36:41.188 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 642 [ CSSD]2013-05-23 17:36:41.188 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:36:41.391 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:36:41.391 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0cf710) skgpid ospid 676 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:42.436 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:42.437 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:42.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:42.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:43.179 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:44.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:44.492 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:45.184 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:45.446 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:36:45.446 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 643 (0x7f431bfd3190) [ CSSD]2013-05-23 17:36:45.446 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3190 (RPC#643) tag(283002d) sent to node 1 [ CSSD]2013-05-23 17:36:45.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:45.447 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 643 (0x7f431bfd3190) [ CSSD]2013-05-23 17:36:45.447 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3190 (RPC#643) state 6, flags 0x202 [ CSSD]2013-05-23 17:36:45.447 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3190, ret 0, client (nil) [ CSSD]2013-05-23 17:36:45.447 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 643 [ CSSD]2013-05-23 17:36:46.439 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:46.439 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:36:46.502 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:46.502 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:47.190 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:48.508 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:48.508 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:49.201 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:50.519 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:50.519 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:51.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:51.107 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015349, gin 1, member 0 [ CSSD]2013-05-23 17:36:51.107 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:36:51.107 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:36:51.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:51.113 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:36:51.113 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:36:51.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:51.442 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:51.442 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:52.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:52.525 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:53.218 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:54.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:54.531 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:55.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:55.472 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:36:55.472 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 644 (0x7f431bfd3210) [ CSSD]2013-05-23 17:36:55.473 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3210 (RPC#644) tag(284002d) sent to node 1 [ CSSD]2013-05-23 17:36:55.473 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:55.473 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 644 (0x7f431bfd3210) [ CSSD]2013-05-23 17:36:55.473 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3210 (RPC#644) state 6, flags 0x202 [ CSSD]2013-05-23 17:36:55.473 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3210, ret 0, client (nil) [ CSSD]2013-05-23 17:36:55.473 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 644 [ CSSD]2013-05-23 17:36:56.422 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:36:56.423 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:36:56.423 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:36:56.423 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(820) version 11:1:1:4 [ CSSD]2013-05-23 17:36:56.429 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:36:56.429 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:56.429 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:36:56.429 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:56.429 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 645 (0x7f431bfd3290) [ CSSD]2013-05-23 17:36:56.430 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3290 (RPC#645) tag(285002d) sent to node 1 [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0285002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015350, gin 1, member 0 [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3290 (RPC#645) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3290, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:36:56.430 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:36:56.431 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 645 [ CSSD]2013-05-23 17:36:56.437 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:36:56.437 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:56.437 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:36:56.437 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 646 (0x7f431bfd3310) [ CSSD]2013-05-23 17:36:56.437 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3310 (RPC#646) tag(286002d) sent to node 1 [ CSSD]2013-05-23 17:36:56.437 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0286002d) incarnation(174631051) [ CSSD]2013-05-23 17:36:56.437 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:36:56.437 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3310 (RPC#646) state 6, flags 0x100 [ CSSD]2013-05-23 17:36:56.437 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3310, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:36:56.438 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 646 [ CSSD]2013-05-23 17:36:56.438 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:36:56.445 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:36:56.445 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:36:56.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:56.532 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:56.640 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:36:56.640 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 820 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:36:57.229 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:36:58.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:36:58.557 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:36:59.232 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:00.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:00.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:01.238 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:01.448 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:01.448 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:02.596 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:02.596 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:03.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:04.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:04.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:05.266 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:05.497 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:37:05.498 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 647 (0x7f431bfd3390) [ CSSD]2013-05-23 17:37:05.498 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3390 (RPC#647) tag(287002d) sent to node 1 [ CSSD]2013-05-23 17:37:05.498 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:05.498 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 647 (0x7f431bfd3390) [ CSSD]2013-05-23 17:37:05.498 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3390 (RPC#647) state 6, flags 0x202 [ CSSD]2013-05-23 17:37:05.498 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3390, ret 0, client (nil) [ CSSD]2013-05-23 17:37:05.499 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 647 [ CSSD]2013-05-23 17:37:06.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:06.366 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015351, gin 1, member 0 [ CSSD]2013-05-23 17:37:06.366 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:06.366 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:06.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:06.373 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:06.373 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:06.451 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:06.451 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:06.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:06.612 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:07.272 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:08.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:08.618 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:09.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:10.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:10.630 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:11.328 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:11.454 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:11.454 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:11.671 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:37:11.672 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:37:11.672 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:37:11.672 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c082370) pid(967) version 11:1:1:4 [ CSSD]2013-05-23 17:37:11.689 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:37:11.689 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:37:11.689 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:37:11.689 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:11.689 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 648 (0x7f431bfd3410) [ CSSD]2013-05-23 17:37:11.689 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3410 (RPC#648) tag(288002d) sent to node 1 [ CSSD]2013-05-23 17:37:11.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0288002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:11.689 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015352, gin 1, member 0 [ CSSD]2013-05-23 17:37:11.689 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:37:11.690 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:37:11.690 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:37:11.690 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3410 (RPC#648) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:11.690 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3410, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:37:11.690 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:37:11.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 648 [ CSSD]2013-05-23 17:37:11.696 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:37:11.696 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:11.696 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:11.696 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 649 (0x7f431bfd3490) [ CSSD]2013-05-23 17:37:11.696 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3490 (RPC#649) tag(289002d) sent to node 1 [ CSSD]2013-05-23 17:37:11.697 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0289002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:11.697 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:11.697 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3490 (RPC#649) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:11.697 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3490, ret 0, client 0x7f431c109f80 member 0x7f431c083a00 [ CSSD]2013-05-23 17:37:11.697 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 649 [ CSSD]2013-05-23 17:37:11.697 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:37:11.899 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:37:11.899 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0cf710) skgpid ospid 967 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:12.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:12.635 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:13.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:14.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:14.636 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015353, gin 1, member 0 [ CSSD]2013-05-23 17:37:14.636 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:14.636 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:14.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:14.642 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:14.642 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:14.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:14.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:15.445 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:15.521 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:37:15.521 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 650 (0x7f431bfd3510) [ CSSD]2013-05-23 17:37:15.522 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3510 (RPC#650) tag(28a002d) sent to node 1 [ CSSD]2013-05-23 17:37:15.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:15.522 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 650 (0x7f431bfd3510) [ CSSD]2013-05-23 17:37:15.522 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3510 (RPC#650) state 6, flags 0x202 [ CSSD]2013-05-23 17:37:15.522 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3510, ret 0, client (nil) [ CSSD]2013-05-23 17:37:15.522 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 650 [ CSSD]2013-05-23 17:37:16.457 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:16.457 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:16.733 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:16.733 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:17.450 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:18.740 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:18.740 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:19.504 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:20.550 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:37:20.551 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:37:20.552 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:37:20.552 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(1308) version 11:1:1:4 [ CSSD]2013-05-23 17:37:20.673 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0922e0) allocated [ CSSD]2013-05-23 17:37:20.674 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:37:20.674 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0922e0 - 1,2,3 [ CSSD]2013-05-23 17:37:20.674 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c0922e0) pid(1318) version 11:1:1:4 [ CSSD]2013-05-23 17:37:20.700 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:37:20.702 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:37:20.704 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0922e0 [ CSSD]2013-05-23 17:37:20.704 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0922e0) con(0x7f431c13f020) skgpid ospid 1318 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:20.773 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(82/0x7f431c135120) [ CSSD]2013-05-23 17:37:20.773 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 82 (0x7f431c135120) [ CSSD]2013-05-23 17:37:20.773 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:37:20.773 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:37:20.773 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 651 (0x7f431bfd3590) [ CSSD]2013-05-23 17:37:20.773 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3590 (RPC#651) tag(28b002d) sent to node 1 [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(028b002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015354, gin 1, member 0 [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c082570/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3590 (RPC#651) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3590, ret 0, client 0x7f431c135120 member 0x7f431c1299e0 [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:37:20.774 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 651 [ CSSD]2013-05-23 17:37:20.782 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 82 (0x7f431c135120) [ CSSD]2013-05-23 17:37:20.782 [1191196992] >TRACE: clssgmExitGrock: client 82 (0x7f431c135120), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:37:20.782 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1299e0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:37:20.782 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 652 (0x7f431bfd3610) [ CSSD]2013-05-23 17:37:20.782 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3610 (RPC#652) tag(28c002d) sent to node 1 [ CSSD]2013-05-23 17:37:20.783 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(028c002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:20.783 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:37:20.783 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3610 (RPC#652) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:20.783 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3610, ret 0, client 0x7f431c135120 member 0x7f431c1299e0 [ CSSD]2013-05-23 17:37:20.783 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 652 [ CSSD]2013-05-23 17:37:20.783 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c1299e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:37:20.784 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:37:20.784 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 1308 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:20.822 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:20.822 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:21.459 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:21.460 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:21.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:21.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:21.616 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015355, gin 1, member 0 [ CSSD]2013-05-23 17:37:21.616 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:21.616 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:21.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:21.622 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:21.622 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:22.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:22.796 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:23.541 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:24.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:24.808 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:25.546 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:37:25.546 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 653 (0x7f431bfd3690) [ CSSD]2013-05-23 17:37:25.546 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3690 (RPC#653) tag(28d002d) sent to node 1 [ CSSD]2013-05-23 17:37:25.547 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:25.547 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:25.547 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 653 (0x7f431bfd3690) [ CSSD]2013-05-23 17:37:25.547 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3690 (RPC#653) state 6, flags 0x202 [ CSSD]2013-05-23 17:37:25.547 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3690, ret 0, client (nil) [ CSSD]2013-05-23 17:37:25.547 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 653 [ CSSD]2013-05-23 17:37:26.462 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:26.462 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:26.832 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:26.832 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:26.935 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:37:26.936 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:37:26.936 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:37:26.936 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(1374) version 11:1:1:4 [ CSSD]2013-05-23 17:37:26.942 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:37:26.942 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:37:26.942 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:37:26.942 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:26.942 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 654 (0x7f431bfd3710) [ CSSD]2013-05-23 17:37:26.942 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3710 (RPC#654) tag(28e002d) sent to node 1 [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(028e002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015356, gin 1, member 0 [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3710 (RPC#654) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3710, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:37:26.943 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 654 [ CSSD]2013-05-23 17:37:26.949 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:37:26.949 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:26.949 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:26.949 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 655 (0x7f431bfd3790) [ CSSD]2013-05-23 17:37:26.949 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3790 (RPC#655) tag(28f002d) sent to node 1 [ CSSD]2013-05-23 17:37:26.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(028f002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:26.949 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:26.950 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3790 (RPC#655) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:26.950 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3790, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:37:26.950 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 655 [ CSSD]2013-05-23 17:37:26.950 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:37:27.152 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:37:27.152 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 1374 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:27.552 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:28.839 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:28.839 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:29.604 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:30.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:30.891 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:31.465 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:31.465 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:31.610 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:32.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:32.897 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:33.702 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:34.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:34.908 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:35.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:35.394 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015357, gin 1, member 0 [ CSSD]2013-05-23 17:37:35.394 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:35.394 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:35.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:35.401 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:35.401 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:35.574 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:37:35.575 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 656 (0x7f431bfd3810) [ CSSD]2013-05-23 17:37:35.575 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3810 (RPC#656) tag(290002d) sent to node 1 [ CSSD]2013-05-23 17:37:35.575 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:35.575 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 656 (0x7f431bfd3810) [ CSSD]2013-05-23 17:37:35.576 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3810 (RPC#656) state 6, flags 0x202 [ CSSD]2013-05-23 17:37:35.576 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3810, ret 0, client (nil) [ CSSD]2013-05-23 17:37:35.576 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 656 [ CSSD]2013-05-23 17:37:35.727 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:36.468 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:36.468 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:36.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:36.863 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015358, gin 1, member 0 [ CSSD]2013-05-23 17:37:36.863 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:36.863 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:36.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:36.869 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:36.869 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:36.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:36.913 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:37.733 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:38.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:38.940 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:39.288 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:37:39.289 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:37:39.290 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:37:39.290 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c082370) pid(1497) version 11:1:1:4 [ CSSD]2013-05-23 17:37:39.296 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x205ecb0) [ CSSD]2013-05-23 17:37:39.296 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x205ecb0) [ CSSD]2013-05-23 17:37:39.296 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x205ecb0 with con 0x230f280, requested num -1 [ CSSD]2013-05-23 17:37:39.296 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:39.296 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 657 (0x7f431bfd3890) [ CSSD]2013-05-23 17:37:39.296 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3890 (RPC#657) tag(291002d) sent to node 1 [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0291002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015359, gin 1, member 0 [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3890 (RPC#657) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3890, ret 0, client 0x205ecb0 member 0x22d7cd0 [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x205ecb0 [ CSSD]2013-05-23 17:37:39.297 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 657 [ CSSD]2013-05-23 17:37:39.304 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x205ecb0) [ CSSD]2013-05-23 17:37:39.304 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x205ecb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:39.304 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7cd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:39.304 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 658 (0x7f431bfd3910) [ CSSD]2013-05-23 17:37:39.304 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3910 (RPC#658) tag(292002d) sent to node 1 [ CSSD]2013-05-23 17:37:39.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0292002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:39.304 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:39.304 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3910 (RPC#658) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:39.304 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3910, ret 0, client 0x205ecb0 member 0x22d7cd0 [ CSSD]2013-05-23 17:37:39.304 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 658 [ CSSD]2013-05-23 17:37:39.304 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7cd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:37:39.507 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:37:39.507 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0840c0) skgpid ospid 1497 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:39.743 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:40.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:40.992 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:41.471 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:41.471 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:41.749 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:42.189 [1191196992] >TRACE: clssgmAllocProc: (0x231d6c0) allocated [ CSSD]2013-05-23 17:37:42.189 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:37:42.190 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231d6c0 - 1,2,3 [ CSSD]2013-05-23 17:37:42.190 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x231d6c0) pid(1532) version 11:1:1:4 [ CSSD]2013-05-23 17:37:42.196 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231d6c0), client(1/0x22e01e0) [ CSSD]2013-05-23 17:37:42.196 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 17:37:42.196 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e01e0 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 17:37:42.196 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:42.196 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 659 (0x7f431bfd3990) [ CSSD]2013-05-23 17:37:42.196 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3990 (RPC#659) tag(293002d) sent to node 1 [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0293002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015360, gin 1, member 0 [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3990 (RPC#659) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3990, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e01e0 [ CSSD]2013-05-23 17:37:42.197 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 659 [ CSSD]2013-05-23 17:37:42.203 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 17:37:42.203 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e01e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:42.203 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:42.203 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 660 (0x7f431bfd3a10) [ CSSD]2013-05-23 17:37:42.203 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3a10 (RPC#660) tag(294002d) sent to node 1 [ CSSD]2013-05-23 17:37:42.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0294002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:42.204 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:42.204 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3a10 (RPC#660) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:42.204 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3a10, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 17:37:42.204 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 660 [ CSSD]2013-05-23 17:37:42.204 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:37:42.406 [1191196992] >TRACE: clssgmDeadProc: proc 0x231d6c0 [ CSSD]2013-05-23 17:37:42.406 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231d6c0) con(0x2312880) skgpid ospid 1532 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:42.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:42.998 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:43.802 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:44.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:44.152 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015361, gin 1, member 0 [ CSSD]2013-05-23 17:37:44.152 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:44.152 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:44.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:44.158 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:44.158 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:45.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:45.009 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:45.602 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:37:45.602 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 661 (0x7f431bfd3a90) [ CSSD]2013-05-23 17:37:45.603 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3a90 (RPC#661) tag(295002d) sent to node 1 [ CSSD]2013-05-23 17:37:45.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:45.603 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 661 (0x7f431bfd3a90) [ CSSD]2013-05-23 17:37:45.603 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3a90 (RPC#661) state 6, flags 0x202 [ CSSD]2013-05-23 17:37:45.603 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3a90, ret 0, client (nil) [ CSSD]2013-05-23 17:37:45.603 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 661 [ CSSD]2013-05-23 17:37:45.808 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:46.474 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:46.474 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:47.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:47.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:47.834 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:49.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:49.040 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:49.846 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:51.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:51.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:51.477 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:51.477 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:37:51.852 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:52.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:52.121 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015362, gin 1, member 0 [ CSSD]2013-05-23 17:37:52.121 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:37:52.121 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:37:52.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:52.128 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:37:52.128 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:37:53.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:53.138 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:53.903 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:55.190 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:55.190 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:55.479 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:37:55.479 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:37:55.628 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:37:55.628 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 662 (0x7f431bfd3b10) [ CSSD]2013-05-23 17:37:55.628 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3b10 (RPC#662) tag(296002d) sent to node 1 [ CSSD]2013-05-23 17:37:55.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:55.629 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 662 (0x7f431bfd3b10) [ CSSD]2013-05-23 17:37:55.629 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3b10 (RPC#662) state 6, flags 0x202 [ CSSD]2013-05-23 17:37:55.629 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3b10, ret 0, client (nil) [ CSSD]2013-05-23 17:37:55.629 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 662 [ CSSD]2013-05-23 17:37:55.909 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:57.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:57.195 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:37:57.438 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 17:37:57.440 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:37:57.440 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 17:37:57.440 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22fc9c0) pid(1783) version 11:1:1:4 [ CSSD]2013-05-23 17:37:57.446 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22fc9c0), client(1/0x2312950) [ CSSD]2013-05-23 17:37:57.446 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:37:57.446 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 17:37:57.446 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:57.446 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 663 (0x7f431bfd3b90) [ CSSD]2013-05-23 17:37:57.446 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3b90 (RPC#663) tag(297002d) sent to node 1 [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0297002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015363, gin 1, member 0 [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3b90 (RPC#663) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3b90, ret 0, client 0x2312950 member 0x22fdb60 [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:37:57.447 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 663 [ CSSD]2013-05-23 17:37:57.453 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:37:57.453 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:57.453 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:37:57.453 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 664 (0x7f431bfd3c10) [ CSSD]2013-05-23 17:37:57.453 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3c10 (RPC#664) tag(298002d) sent to node 1 [ CSSD]2013-05-23 17:37:57.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0298002d) incarnation(174631051) [ CSSD]2013-05-23 17:37:57.453 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:37:57.453 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3c10 (RPC#664) state 6, flags 0x100 [ CSSD]2013-05-23 17:37:57.453 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3c10, ret 0, client 0x2312950 member 0x22fdb60 [ CSSD]2013-05-23 17:37:57.454 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 664 [ CSSD]2013-05-23 17:37:57.454 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:37:57.656 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 17:37:57.656 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x205ecb0) skgpid ospid 1783 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:37:57.954 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:37:59.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:37:59.201 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:00.026 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:00.482 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:00.482 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:01.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:01.211 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:02.033 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:03.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:03.236 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:04.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:05.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:05.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:05.485 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:05.485 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:05.651 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:38:05.651 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 665 (0x7f431bfd3c90) [ CSSD]2013-05-23 17:38:05.651 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3c90 (RPC#665) tag(299002d) sent to node 1 [ CSSD]2013-05-23 17:38:05.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:05.652 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 665 (0x7f431bfd3c90) [ CSSD]2013-05-23 17:38:05.652 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3c90 (RPC#665) state 6, flags 0x202 [ CSSD]2013-05-23 17:38:05.652 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3c90, ret 0, client (nil) [ CSSD]2013-05-23 17:38:05.652 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 665 [ CSSD]2013-05-23 17:38:06.048 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:07.265 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:07.265 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:07.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:07.371 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015364, gin 1, member 0 [ CSSD]2013-05-23 17:38:07.371 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:38:07.371 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:38:07.379 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:07.379 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:38:07.379 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:38:08.054 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:09.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:09.270 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:10.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:10.488 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:10.488 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:11.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:11.273 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:12.133 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:12.698 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:38:12.699 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:38:12.700 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:38:12.700 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c082370) pid(1937) version 11:1:1:4 [ CSSD]2013-05-23 17:38:12.716 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:38:12.716 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:38:12.716 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:38:12.716 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:12.716 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 666 (0x7f431bfd3d10) [ CSSD]2013-05-23 17:38:12.716 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3d10 (RPC#666) tag(29a002d) sent to node 1 [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(029a002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015365, gin 1, member 0 [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3d10 (RPC#666) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3d10, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:38:12.717 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 666 [ CSSD]2013-05-23 17:38:12.723 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:38:12.723 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:12.723 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:12.723 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 667 (0x7f431bfd3d90) [ CSSD]2013-05-23 17:38:12.723 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3d90 (RPC#667) tag(29b002d) sent to node 1 [ CSSD]2013-05-23 17:38:12.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(029b002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:12.724 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:12.724 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3d90 (RPC#667) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:12.724 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3d90, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:38:12.724 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 667 [ CSSD]2013-05-23 17:38:12.724 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:38:12.926 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:38:12.926 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0cf710) skgpid ospid 1937 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:13.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:13.278 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:14.224 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:15.311 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:15.311 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:15.491 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:15.491 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:15.681 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:38:15.682 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 668 (0x7f431bfd3e10) [ CSSD]2013-05-23 17:38:15.682 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3e10 (RPC#668) tag(29c002d) sent to node 1 [ CSSD]2013-05-23 17:38:15.682 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:15.682 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 668 (0x7f431bfd3e10) [ CSSD]2013-05-23 17:38:15.682 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3e10 (RPC#668) state 6, flags 0x202 [ CSSD]2013-05-23 17:38:15.682 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3e10, ret 0, client (nil) [ CSSD]2013-05-23 17:38:15.682 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 668 [ CSSD]2013-05-23 17:38:16.271 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:17.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:17.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:18.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:19.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:19.362 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:19.493 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:19.494 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:38:20.307 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:21.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:21.413 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:22.313 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:22.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:22.623 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015366, gin 1, member 0 [ CSSD]2013-05-23 17:38:22.623 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:38:22.623 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:38:22.629 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:22.629 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:38:22.629 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:38:23.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:23.419 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:24.315 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:24.497 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:24.497 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:25.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:25.431 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:25.705 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:38:25.706 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 669 (0x7f431bfd3e90) [ CSSD]2013-05-23 17:38:25.706 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3e90 (RPC#669) tag(29d002d) sent to node 1 [ CSSD]2013-05-23 17:38:25.706 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:25.706 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 669 (0x7f431bfd3e90) [ CSSD]2013-05-23 17:38:25.706 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3e90 (RPC#669) state 6, flags 0x202 [ CSSD]2013-05-23 17:38:25.706 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd3e90, ret 0, client (nil) [ CSSD]2013-05-23 17:38:25.706 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 669 [ CSSD]2013-05-23 17:38:26.320 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:27.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:27.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:27.958 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:38:27.959 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:38:27.959 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:38:27.959 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c082370) pid(2073) version 11:1:1:4 [ CSSD]2013-05-23 17:38:27.965 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:38:27.965 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:38:27.965 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:38:27.965 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:27.965 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 670 (0x7f431bfd3f10) [ CSSD]2013-05-23 17:38:27.965 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3f10 (RPC#670) tag(29e002d) sent to node 1 [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(029e002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015367, gin 1, member 0 [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c083700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3f10 (RPC#670) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd3f10, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:38:27.966 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 670 [ CSSD]2013-05-23 17:38:27.972 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:38:27.972 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:27.972 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c083a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:27.972 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 671 (0x7f431bfd3f90) [ CSSD]2013-05-23 17:38:27.972 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd3f90 (RPC#671) tag(29f002d) sent to node 1 [ CSSD]2013-05-23 17:38:27.972 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(029f002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:27.972 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:27.972 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd3f90 (RPC#671) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:27.972 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd3f90, ret 0, client 0x7f431c135120 member 0x7f431c083a00 [ CSSD]2013-05-23 17:38:27.972 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 671 [ CSSD]2013-05-23 17:38:27.972 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c083a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:38:28.175 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:38:28.175 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0e5200) skgpid ospid 2073 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:28.333 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:29.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:29.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:29.499 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:29.499 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:30.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:31.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:31.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:32.382 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:33.460 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:33.460 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:34.415 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:34.502 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:34.502 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:35.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:35.511 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:35.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:35.653 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015368, gin 1, member 0 [ CSSD]2013-05-23 17:38:35.653 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:38:35.653 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:38:35.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:35.660 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:38:35.660 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:38:35.731 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:38:35.731 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 672 (0x7f431bfd4010) [ CSSD]2013-05-23 17:38:35.731 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4010 (RPC#672) tag(2a0002d) sent to node 1 [ CSSD]2013-05-23 17:38:35.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:35.732 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 672 (0x7f431bfd4010) [ CSSD]2013-05-23 17:38:35.732 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4010 (RPC#672) state 6, flags 0x202 [ CSSD]2013-05-23 17:38:35.732 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4010, ret 0, client (nil) [ CSSD]2013-05-23 17:38:35.732 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 672 [ CSSD]2013-05-23 17:38:36.421 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:37.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:37.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:37.588 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:38:37.588 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:38:37.589 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:38:37.589 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c082370) pid(1704/1704) version 11:1:1:4 [ CSSD]2013-05-23 17:38:37.605 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:38:37.605 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:38:37.605 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:38:37.605 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:38:37.878 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:37.878 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015369, gin 1, member 0 [ CSSD]2013-05-23 17:38:37.878 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:38:37.878 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:38:37.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:37.885 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:38:37.885 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x230f170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:38:38.433 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:39.505 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:39.505 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:39.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:39.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:39.546 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134070) allocated [ CSSD]2013-05-23 17:38:39.547 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:38:39.547 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134070 - 1,2,3 [ CSSD]2013-05-23 17:38:39.547 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c134070) pid(2201) version 11:1:1:4 [ CSSD]2013-05-23 17:38:39.553 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c134070), client(1/0x205ecb0) [ CSSD]2013-05-23 17:38:39.553 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x205ecb0) [ CSSD]2013-05-23 17:38:39.553 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x205ecb0 with con 0x230f280, requested num -1 [ CSSD]2013-05-23 17:38:39.553 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:39.553 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 673 (0x7f431bfd4090) [ CSSD]2013-05-23 17:38:39.553 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4090 (RPC#673) tag(2a1002d) sent to node 1 [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02a1002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015370, gin 1, member 0 [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4090 (RPC#673) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4090, ret 0, client 0x205ecb0 member 0x22fdb60 [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x205ecb0 [ CSSD]2013-05-23 17:38:39.554 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 673 [ CSSD]2013-05-23 17:38:39.560 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x205ecb0) [ CSSD]2013-05-23 17:38:39.560 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x205ecb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:39.560 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:39.560 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 674 (0x7f431bfd4110) [ CSSD]2013-05-23 17:38:39.560 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4110 (RPC#674) tag(2a2002d) sent to node 1 [ CSSD]2013-05-23 17:38:39.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02a2002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:39.560 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:39.560 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4110 (RPC#674) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:39.560 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4110, ret 0, client 0x205ecb0 member 0x22fdb60 [ CSSD]2013-05-23 17:38:39.560 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 674 [ CSSD]2013-05-23 17:38:39.560 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:38:39.763 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134070 [ CSSD]2013-05-23 17:38:39.763 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134070) con(0x7f431c13f020) skgpid ospid 2201 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:40.439 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:41.535 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:41.535 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:42.443 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:43.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:43.168 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015371, gin 1, member 0 [ CSSD]2013-05-23 17:38:43.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:38:43.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:38:43.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:43.174 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:38:43.174 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:38:43.204 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:38:43.205 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:38:43.206 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:38:43.206 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(2248) version 11:1:1:4 [ CSSD]2013-05-23 17:38:43.211 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 17:38:43.211 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:38:43.211 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 17:38:43.211 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:43.211 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 675 (0x7f431bfd4190) [ CSSD]2013-05-23 17:38:43.211 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4190 (RPC#675) tag(2a3002d) sent to node 1 [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02a3002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015372, gin 1, member 0 [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4190 (RPC#675) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4190, ret 0, client 0x2312950 member 0x22fdb60 [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:38:43.212 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 675 [ CSSD]2013-05-23 17:38:43.217 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:38:43.217 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:43.217 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fdb60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:43.217 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 676 (0x7f431bfd4210) [ CSSD]2013-05-23 17:38:43.218 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4210 (RPC#676) tag(2a4002d) sent to node 1 [ CSSD]2013-05-23 17:38:43.218 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02a4002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:43.218 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:43.218 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4210 (RPC#676) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:43.218 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4210, ret 0, client 0x2312950 member 0x22fdb60 [ CSSD]2013-05-23 17:38:43.218 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 676 [ CSSD]2013-05-23 17:38:43.218 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fdb60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:38:43.298 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:38:43.299 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:38:43.300 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:38:43.300 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:38:43.301 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:38:43.302 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:38:43.420 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:38:43.420 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 2248 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:43.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:43.540 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:44.455 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:44.508 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:44.508 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:45.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:45.612 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:45.762 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:38:45.762 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 677 (0x7f431bfd4290) [ CSSD]2013-05-23 17:38:45.762 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4290 (RPC#677) tag(2a5002d) sent to node 1 [ CSSD]2013-05-23 17:38:45.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:45.763 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 677 (0x7f431bfd4290) [ CSSD]2013-05-23 17:38:45.763 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4290 (RPC#677) state 6, flags 0x202 [ CSSD]2013-05-23 17:38:45.763 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4290, ret 0, client (nil) [ CSSD]2013-05-23 17:38:45.763 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 677 [ CSSD]2013-05-23 17:38:46.501 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:47.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:47.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:48.513 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:49.510 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:49.511 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:49.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:49.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:50.519 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:51.655 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:51.655 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:52.525 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:53.138 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:53.138 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015373, gin 1, member 0 [ CSSD]2013-05-23 17:38:53.138 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:38:53.138 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:38:53.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:53.144 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:38:53.144 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:38:53.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:53.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:54.514 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:54.514 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:54.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:55.083 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c094e90) [ CSSD]2013-05-23 17:38:55.083 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c094e90), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:38:55.083 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c094e90), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:38:55.083 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:38:55.084 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135e90 [ CSSD]2013-05-23 17:38:55.084 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135e90) con(0x7f431c132f30) skgpid 29500 ospid 29500 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:55.086 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c0e5910) [ CSSD]2013-05-23 17:38:55.086 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c0e5910), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:38:55.086 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c0e5910), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:38:55.086 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 22 [ CSSD]2013-05-23 17:38:55.096 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:38:55.096 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c13f730), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:38:55.096 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c13f730), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:38:55.096 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 17:38:55.096 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c048500 [ CSSD]2013-05-23 17:38:55.096 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c048500) con(0x7f431c108e60) skgpid 29502 ospid 29502 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:55.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:55.712 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:55.788 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:38:55.788 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 678 (0x7f431bfd4310) [ CSSD]2013-05-23 17:38:55.788 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4310 (RPC#678) tag(2a6002d) sent to node 1 [ CSSD]2013-05-23 17:38:55.789 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:55.789 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 678 (0x7f431bfd4310) [ CSSD]2013-05-23 17:38:55.789 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4310 (RPC#678) state 6, flags 0x202 [ CSSD]2013-05-23 17:38:55.789 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4310, ret 0, client (nil) [ CSSD]2013-05-23 17:38:55.789 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 678 [ CSSD]2013-05-23 17:38:56.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:57.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:57.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:38:58.455 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 17:38:58.456 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:38:58.456 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 17:38:58.456 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22fc9c0) pid(2404) version 11:1:1:4 [ CSSD]2013-05-23 17:38:58.462 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22fc9c0), client(1/0x2312950) [ CSSD]2013-05-23 17:38:58.462 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:38:58.462 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 17:38:58.462 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:58.462 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 679 (0x7f431bfd4390) [ CSSD]2013-05-23 17:38:58.462 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4390 (RPC#679) tag(2a7002d) sent to node 1 [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02a7002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015374, gin 1, member 0 [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4390 (RPC#679) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4390, ret 0, client 0x2312950 member 0x231d6c0 [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:38:58.463 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 679 [ CSSD]2013-05-23 17:38:58.469 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:38:58.469 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:58.469 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231d6c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:38:58.469 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 680 (0x7f431bfd4410) [ CSSD]2013-05-23 17:38:58.469 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4410 (RPC#680) tag(2a8002d) sent to node 1 [ CSSD]2013-05-23 17:38:58.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02a8002d) incarnation(174631051) [ CSSD]2013-05-23 17:38:58.470 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:38:58.470 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4410 (RPC#680) state 6, flags 0x100 [ CSSD]2013-05-23 17:38:58.470 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4410, ret 0, client 0x2312950 member 0x231d6c0 [ CSSD]2013-05-23 17:38:58.470 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 680 [ CSSD]2013-05-23 17:38:58.470 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231d6c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:38:58.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:38:58.673 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 17:38:58.673 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x205ecb0) skgpid ospid 2404 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:38:59.516 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:38:59.517 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:38:59.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:38:59.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:00.560 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:01.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:01.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:02.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:03.519 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:03.519 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:39:03.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:03.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:04.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:05.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:05.813 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:05.813 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:39:05.813 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 681 (0x7f431bfd4490) [ CSSD]2013-05-23 17:39:05.813 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4490 (RPC#681) tag(2a9002d) sent to node 1 [ CSSD]2013-05-23 17:39:05.814 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:05.814 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 681 (0x7f431bfd4490) [ CSSD]2013-05-23 17:39:05.814 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4490 (RPC#681) state 6, flags 0x202 [ CSSD]2013-05-23 17:39:05.814 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4490, ret 0, client (nil) [ CSSD]2013-05-23 17:39:05.814 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 681 [ CSSD]2013-05-23 17:39:06.642 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:07.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:07.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:08.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:08.394 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015375, gin 1, member 0 [ CSSD]2013-05-23 17:39:08.394 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:39:08.394 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:39:08.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:08.400 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:39:08.401 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:39:08.522 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:08.522 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:08.654 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:09.829 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:09.829 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:10.660 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:11.343 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:39:11.344 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:39:11.344 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:39:11.344 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22dacd0) pid(2382/2382) version 11:1:1:4 [ CSSD]2013-05-23 17:39:11.360 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x22d7cd0) [ CSSD]2013-05-23 17:39:11.360 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22d7cd0) [ CSSD]2013-05-23 17:39:11.360 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:39:11.360 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 17:39:11.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:11.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:12.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:13.525 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:13.526 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:13.707 [1191196992] >TRACE: clssgmAllocProc: (0x22f8740) allocated [ CSSD]2013-05-23 17:39:13.708 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:39:13.708 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8740 - 1,2,3 [ CSSD]2013-05-23 17:39:13.708 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7750) proc(0x22f8740) pid(2552) version 11:1:1:4 [ CSSD]2013-05-23 17:39:13.714 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f8740), client(1/0x22dff40) [ CSSD]2013-05-23 17:39:13.714 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dff40) [ CSSD]2013-05-23 17:39:13.714 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dff40 with con 0x22f7cd0, requested num -1 [ CSSD]2013-05-23 17:39:13.714 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:13.714 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 682 (0x7f431bfd4510) [ CSSD]2013-05-23 17:39:13.714 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4510 (RPC#682) tag(2aa002d) sent to node 1 [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02aa002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015376, gin 1, member 0 [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x205e170/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4510 (RPC#682) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4510, ret 0, client 0x22dff40 member 0x22ff740 [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dff40 [ CSSD]2013-05-23 17:39:13.715 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 682 [ CSSD]2013-05-23 17:39:13.721 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dff40) [ CSSD]2013-05-23 17:39:13.721 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dff40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:13.721 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22ff740) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:13.721 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 683 (0x7f431bfd4590) [ CSSD]2013-05-23 17:39:13.721 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4590 (RPC#683) tag(2ab002d) sent to node 1 [ CSSD]2013-05-23 17:39:13.722 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ab002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:13.722 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:13.722 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4590 (RPC#683) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:13.722 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4590, ret 0, client 0x22dff40 member 0x22ff740 [ CSSD]2013-05-23 17:39:13.722 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 683 [ CSSD]2013-05-23 17:39:13.722 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22ff740) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:39:13.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:13.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:13.924 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f8740 [ CSSD]2013-05-23 17:39:13.924 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f8740) con(0x22d7750) skgpid ospid 2552 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:39:14.717 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:15.857 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:39:15.858 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 684 (0x7f431bfd4610) [ CSSD]2013-05-23 17:39:15.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4610 (RPC#684) tag(2ac002d) sent to node 1 [ CSSD]2013-05-23 17:39:15.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:15.858 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 684 (0x7f431bfd4610) [ CSSD]2013-05-23 17:39:15.858 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4610 (RPC#684) state 6, flags 0x202 [ CSSD]2013-05-23 17:39:15.858 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4610, ret 0, client (nil) [ CSSD]2013-05-23 17:39:15.859 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 684 [ CSSD]2013-05-23 17:39:15.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:15.913 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:16.763 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:17.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:17.920 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:18.529 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:18.529 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:18.815 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:19.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:19.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:20.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:21.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:21.957 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:22.827 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:23.531 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:23.532 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:23.644 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:23.644 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015377, gin 1, member 0 [ CSSD]2013-05-23 17:39:23.644 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:39:23.644 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:39:23.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:23.650 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:39:23.650 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:39:23.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:23.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:24.836 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:25.885 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:39:25.885 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 685 (0x7f431bfd4690) [ CSSD]2013-05-23 17:39:25.885 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4690 (RPC#685) tag(2ad002d) sent to node 1 [ CSSD]2013-05-23 17:39:25.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:25.886 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 685 (0x7f431bfd4690) [ CSSD]2013-05-23 17:39:25.886 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4690 (RPC#685) state 6, flags 0x202 [ CSSD]2013-05-23 17:39:25.886 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4690, ret 0, client (nil) [ CSSD]2013-05-23 17:39:25.886 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 685 [ CSSD]2013-05-23 17:39:25.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:25.983 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:26.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:27.534 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:27.534 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:39:27.989 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:27.989 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:28.168 [1191196992] >TRACE: clssgmAllocProc: (0x22ff670) allocated [ CSSD]2013-05-23 17:39:28.168 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:39:28.169 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ff670 - 1,2,3 [ CSSD]2013-05-23 17:39:28.169 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23037d0) proc(0x22ff670) pid(27626/27626) version 11:1:1:4 [ CSSD]2013-05-23 17:39:28.185 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22ff670), client(1/0x22f7ce0) [ CSSD]2013-05-23 17:39:28.185 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 17:39:28.185 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:39:28.185 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:39:28.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:28.956 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 17:39:28.957 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:39:28.957 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 17:39:28.957 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dff90) proc(0x22d7350) pid(2718) version 11:1:1:4 [ CSSD]2013-05-23 17:39:28.962 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d7350), client(1/0x22f1d30) [ CSSD]2013-05-23 17:39:28.962 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:28.962 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x205e240, requested num -1 [ CSSD]2013-05-23 17:39:28.962 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:28.962 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 686 (0x7f431bfd4710) [ CSSD]2013-05-23 17:39:28.963 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4710 (RPC#686) tag(2ae002d) sent to node 1 [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ae002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015378, gin 1, member 0 [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6cf0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4710 (RPC#686) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4710, ret 0, client 0x22f1d30 member 0x22d7550 [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:39:28.963 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 686 [ CSSD]2013-05-23 17:39:28.969 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:28.969 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:28.969 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7550) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:28.969 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 687 (0x7f431bfd4790) [ CSSD]2013-05-23 17:39:28.969 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4790 (RPC#687) tag(2af002d) sent to node 1 [ CSSD]2013-05-23 17:39:28.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02af002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:28.969 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:28.969 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4790 (RPC#687) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:28.969 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4790, ret 0, client 0x22f1d30 member 0x22d7550 [ CSSD]2013-05-23 17:39:28.970 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 687 [ CSSD]2013-05-23 17:39:28.970 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7550) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:39:29.172 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 17:39:29.172 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x22dff90) skgpid ospid 2718 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:39:29.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:29.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:30.465 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:39:30.466 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:39:30.467 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:39:30.467 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:39:30.548 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:30.548 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015379, gin 1, member 0 [ CSSD]2013-05-23 17:39:30.548 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c0951b0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:39:30.548 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:39:30.556 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:30.556 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:39:30.556 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:39:30.941 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:32.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:32.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:32.537 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:32.537 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:32.947 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:34.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:34.023 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:34.959 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:35.910 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:39:35.911 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 688 (0x7f431bfd4810) [ CSSD]2013-05-23 17:39:35.911 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4810 (RPC#688) tag(2b0002d) sent to node 1 [ CSSD]2013-05-23 17:39:35.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:35.911 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 688 (0x7f431bfd4810) [ CSSD]2013-05-23 17:39:35.911 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4810 (RPC#688) state 6, flags 0x202 [ CSSD]2013-05-23 17:39:35.911 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4810, ret 0, client (nil) [ CSSD]2013-05-23 17:39:35.911 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 688 [ CSSD]2013-05-23 17:39:35.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:35.918 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015380, gin 1, member 0 [ CSSD]2013-05-23 17:39:35.918 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:39:35.918 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:39:35.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:35.925 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:39:35.925 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:39:36.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:36.034 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:36.964 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:37.541 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:37.541 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:38.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:38.059 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:38.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:38.897 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015381, gin 1, member 0 [ CSSD]2013-05-23 17:39:38.897 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:39:38.897 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:39:38.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:38.903 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:39:38.903 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:39:38.986 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:39.804 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 17:39:39.805 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:39:39.805 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 17:39:39.805 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dff90) proc(0x22d7350) pid(2833) version 11:1:1:4 [ CSSD]2013-05-23 17:39:39.812 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d7350), client(1/0x22f1d30) [ CSSD]2013-05-23 17:39:39.812 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:39.812 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x205e240, requested num -1 [ CSSD]2013-05-23 17:39:39.812 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:39.812 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 689 (0x7f431bfd4890) [ CSSD]2013-05-23 17:39:39.812 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4890 (RPC#689) tag(2b1002d) sent to node 1 [ CSSD]2013-05-23 17:39:39.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02b1002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:39.812 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015382, gin 1, member 0 [ CSSD]2013-05-23 17:39:39.812 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6cf0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:39:39.813 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:39:39.813 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:39:39.813 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4890 (RPC#689) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:39.813 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4890, ret 0, client 0x22f1d30 member 0x22d7550 [ CSSD]2013-05-23 17:39:39.813 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:39:39.813 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 689 [ CSSD]2013-05-23 17:39:39.819 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:39.819 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:39.819 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7550) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:39.819 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 690 (0x7f431bfd4910) [ CSSD]2013-05-23 17:39:39.819 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4910 (RPC#690) tag(2b2002d) sent to node 1 [ CSSD]2013-05-23 17:39:39.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02b2002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:39.819 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:39.819 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4910 (RPC#690) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:39.819 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4910, ret 0, client 0x22f1d30 member 0x22d7550 [ CSSD]2013-05-23 17:39:39.819 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 690 [ CSSD]2013-05-23 17:39:39.819 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7550) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:39:40.022 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 17:39:40.022 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x22dff90) skgpid ospid 2833 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:39:40.071 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:40.071 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:40.991 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:42.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:42.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:42.544 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:42.544 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:42.996 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:44.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:44.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:44.203 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 17:39:44.204 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:39:44.205 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 17:39:44.205 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dfee0) proc(0x22d7350) pid(2866) version 11:1:1:4 [ CSSD]2013-05-23 17:39:44.222 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d7350), client(1/0x22f1d30) [ CSSD]2013-05-23 17:39:44.222 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:44.222 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302da0, requested num -1 [ CSSD]2013-05-23 17:39:44.222 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:44.222 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 691 (0x7f431bfd4990) [ CSSD]2013-05-23 17:39:44.222 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4990 (RPC#691) tag(2b3002d) sent to node 1 [ CSSD]2013-05-23 17:39:44.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02b3002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:44.222 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015383, gin 1, member 0 [ CSSD]2013-05-23 17:39:44.222 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d6cf0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:39:44.222 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:39:44.222 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:39:44.223 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4990 (RPC#691) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:44.223 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4990, ret 0, client 0x22f1d30 member 0x2302910 [ CSSD]2013-05-23 17:39:44.223 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:39:44.223 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 691 [ CSSD]2013-05-23 17:39:44.228 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:44.228 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:44.228 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302910) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:44.228 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 692 (0x7f431bfd4a10) [ CSSD]2013-05-23 17:39:44.229 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4a10 (RPC#692) tag(2b4002d) sent to node 1 [ CSSD]2013-05-23 17:39:44.229 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02b4002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:44.229 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:44.229 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4a10 (RPC#692) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:44.229 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4a10, ret 0, client 0x22f1d30 member 0x2302910 [ CSSD]2013-05-23 17:39:44.229 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 692 [ CSSD]2013-05-23 17:39:44.229 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302910) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:39:44.431 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 17:39:44.431 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x22dfee0) skgpid ospid 2866 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:39:45.029 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:45.935 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:39:45.935 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 693 (0x7f431bfd4a90) [ CSSD]2013-05-23 17:39:45.936 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4a90 (RPC#693) tag(2b5002d) sent to node 1 [ CSSD]2013-05-23 17:39:45.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:45.936 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 693 (0x7f431bfd4a90) [ CSSD]2013-05-23 17:39:45.936 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4a90 (RPC#693) state 6, flags 0x202 [ CSSD]2013-05-23 17:39:45.936 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4a90, ret 0, client (nil) [ CSSD]2013-05-23 17:39:45.936 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 693 [ CSSD]2013-05-23 17:39:46.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:46.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:46.546 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:46.546 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:39:47.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:48.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:48.161 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:49.047 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:50.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:50.172 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:50.681 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c082370), client(2/0x2302d30) [ CSSD]2013-05-23 17:39:50.681 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2302d30) [ CSSD]2013-05-23 17:39:50.681 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:39:50.681 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 23 [ CSSD]2013-05-23 17:39:51.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:51.549 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:51.549 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:52.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:52.178 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:53.059 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:54.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:54.148 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015384, gin 1, member 0 [ CSSD]2013-05-23 17:39:54.148 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:39:54.148 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:39:54.155 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:54.155 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:39:54.155 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:39:54.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:54.184 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:55.070 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:55.958 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:39:55.958 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 694 (0x7f431bfd4b10) [ CSSD]2013-05-23 17:39:55.959 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4b10 (RPC#694) tag(2b6002d) sent to node 1 [ CSSD]2013-05-23 17:39:55.959 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:55.959 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 694 (0x7f431bfd4b10) [ CSSD]2013-05-23 17:39:55.959 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4b10 (RPC#694) state 6, flags 0x202 [ CSSD]2013-05-23 17:39:55.959 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4b10, ret 0, client (nil) [ CSSD]2013-05-23 17:39:55.959 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 694 [ CSSD]2013-05-23 17:39:56.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:56.235 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:56.552 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:39:56.552 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:39:57.076 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:58.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:39:58.241 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:39:59.127 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:39:59.463 [1191196992] >TRACE: clssgmAllocProc: (0x2302840) allocated [ CSSD]2013-05-23 17:39:59.465 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:39:59.465 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302840 - 1,2,3 [ CSSD]2013-05-23 17:39:59.465 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e300) proc(0x2302840) pid(3009) version 11:1:1:4 [ CSSD]2013-05-23 17:39:59.470 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x2302840), client(1/0x22f1d30) [ CSSD]2013-05-23 17:39:59.470 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:59.470 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a40, requested num -1 [ CSSD]2013-05-23 17:39:59.470 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:59.470 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 695 (0x7f431bfd4b90) [ CSSD]2013-05-23 17:39:59.471 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4b90 (RPC#695) tag(2b7002d) sent to node 1 [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02b7002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015385, gin 1, member 0 [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4b90 (RPC#695) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4b90, ret 0, client 0x22f1d30 member 0x2302640 [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:39:59.471 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 695 [ CSSD]2013-05-23 17:39:59.478 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:39:59.478 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:59.478 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302640) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:39:59.478 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 696 (0x7f431bfd4c10) [ CSSD]2013-05-23 17:39:59.478 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4c10 (RPC#696) tag(2b8002d) sent to node 1 [ CSSD]2013-05-23 17:39:59.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02b8002d) incarnation(174631051) [ CSSD]2013-05-23 17:39:59.478 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:39:59.478 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4c10 (RPC#696) state 6, flags 0x100 [ CSSD]2013-05-23 17:39:59.478 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4c10, ret 0, client 0x22f1d30 member 0x2302640 [ CSSD]2013-05-23 17:39:59.478 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 696 [ CSSD]2013-05-23 17:39:59.478 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302640) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:39:59.681 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302840 [ CSSD]2013-05-23 17:39:59.681 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302840) con(0x205e300) skgpid ospid 3009 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:00.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:00.253 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:01.153 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:01.556 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:01.556 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:02.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:02.278 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:03.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:04.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:04.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:05.170 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:05.987 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:40:05.987 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 697 (0x7f431bfd4c90) [ CSSD]2013-05-23 17:40:05.987 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4c90 (RPC#697) tag(2b9002d) sent to node 1 [ CSSD]2013-05-23 17:40:05.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:05.988 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 697 (0x7f431bfd4c90) [ CSSD]2013-05-23 17:40:05.988 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4c90 (RPC#697) state 6, flags 0x202 [ CSSD]2013-05-23 17:40:05.988 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4c90, ret 0, client (nil) [ CSSD]2013-05-23 17:40:05.988 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 697 [ CSSD]2013-05-23 17:40:06.335 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:06.335 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:06.559 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:06.559 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:07.176 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:07.977 [1191196992] >TRACE: clssgmAllocProc: (0x2302840) allocated [ CSSD]2013-05-23 17:40:07.978 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:40:07.979 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2302840 - 1,2,3 [ CSSD]2013-05-23 17:40:07.979 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e300) proc(0x2302840) pid(3305) version 11:1:1:4 [ CSSD]2013-05-23 17:40:08.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:08.341 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:08.493 [1191196992] >TRACE: clssgmDeadProc: proc 0x2302840 [ CSSD]2013-05-23 17:40:08.493 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2302840) con(0x205e300) skgpid ospid 3305 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:08.668 [1191196992] >TRACE: clssgmAllocProc: (0x231ed60) allocated [ CSSD]2013-05-23 17:40:08.669 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:40:08.670 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed60 - 1,2,3 [ CSSD]2013-05-23 17:40:08.670 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x231ed60) pid(1702/1702) version 11:1:1:4 [ CSSD]2013-05-23 17:40:08.685 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x231ed60), client(1/0x22f1d30) [ CSSD]2013-05-23 17:40:08.685 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:40:08.685 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:40:08.685 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:40:09.227 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:09.246 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(2/0x22d7210) [ CSSD]2013-05-23 17:40:09.246 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22d7210) [ CSSD]2013-05-23 17:40:09.246 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:40:09.246 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 17:40:09.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:09.398 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015386, gin 1, member 0 [ CSSD]2013-05-23 17:40:09.398 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:40:09.398 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:40:09.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:09.404 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:40:09.404 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:40:10.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:10.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:11.253 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:11.562 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:11.562 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:12.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:12.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:13.345 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:14.364 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:14.364 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:14.713 [1191196992] >TRACE: clssgmAllocProc: (0x22f5000) allocated [ CSSD]2013-05-23 17:40:14.714 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:40:14.715 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f5000 - 1,2,3 [ CSSD]2013-05-23 17:40:14.715 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d6fc0) proc(0x22f5000) pid(3447) version 11:1:1:4 [ CSSD]2013-05-23 17:40:14.731 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f5000), client(1/0x22f2cf0) [ CSSD]2013-05-23 17:40:14.731 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f2cf0) [ CSSD]2013-05-23 17:40:14.731 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f2cf0 with con 0x22f5260, requested num -1 [ CSSD]2013-05-23 17:40:14.731 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:14.731 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 698 (0x7f431bfd4d10) [ CSSD]2013-05-23 17:40:14.732 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4d10 (RPC#698) tag(2ba002d) sent to node 1 [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ba002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015387, gin 1, member 0 [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f2e60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4d10 (RPC#698) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4d10, ret 0, client 0x22f2cf0 member 0x22f4ef0 [ CSSD]2013-05-23 17:40:14.732 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f2cf0 [ CSSD]2013-05-23 17:40:14.733 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 698 [ CSSD]2013-05-23 17:40:14.738 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f2cf0) [ CSSD]2013-05-23 17:40:14.738 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f2cf0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:14.738 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4ef0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:14.738 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 699 (0x7f431bfd4d90) [ CSSD]2013-05-23 17:40:14.738 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4d90 (RPC#699) tag(2bb002d) sent to node 1 [ CSSD]2013-05-23 17:40:14.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02bb002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:14.738 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:14.738 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4d90 (RPC#699) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:14.738 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4d90, ret 0, client 0x22f2cf0 member 0x22f4ef0 [ CSSD]2013-05-23 17:40:14.739 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 699 [ CSSD]2013-05-23 17:40:14.739 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4ef0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:40:14.941 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f5000 [ CSSD]2013-05-23 17:40:14.941 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f5000) con(0x22d6fc0) skgpid ospid 3447 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:15.351 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:16.009 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:40:16.009 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 700 (0x7f431bfd4e10) [ CSSD]2013-05-23 17:40:16.009 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4e10 (RPC#700) tag(2bc002d) sent to node 1 [ CSSD]2013-05-23 17:40:16.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:16.010 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 700 (0x7f431bfd4e10) [ CSSD]2013-05-23 17:40:16.010 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4e10 (RPC#700) state 6, flags 0x202 [ CSSD]2013-05-23 17:40:16.010 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4e10, ret 0, client (nil) [ CSSD]2013-05-23 17:40:16.010 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 700 [ CSSD]2013-05-23 17:40:16.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:16.376 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:16.565 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:16.565 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:17.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:18.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:18.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:19.428 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:20.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:20.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:21.434 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:21.568 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:21.568 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:22.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:22.440 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:23.446 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:24.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:24.446 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:24.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:24.658 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015388, gin 1, member 0 [ CSSD]2013-05-23 17:40:24.658 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:40:24.658 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:40:24.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:24.665 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:40:24.665 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:40:25.452 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:26.032 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:40:26.033 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 701 (0x7f431bfd4e90) [ CSSD]2013-05-23 17:40:26.033 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4e90 (RPC#701) tag(2bd002d) sent to node 1 [ CSSD]2013-05-23 17:40:26.033 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:26.033 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 701 (0x7f431bfd4e90) [ CSSD]2013-05-23 17:40:26.033 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4e90 (RPC#701) state 6, flags 0x202 [ CSSD]2013-05-23 17:40:26.033 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd4e90, ret 0, client (nil) [ CSSD]2013-05-23 17:40:26.034 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 701 [ CSSD]2013-05-23 17:40:26.455 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:26.455 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:26.534 [1191196992] >TRACE: clssgmAllocProc: (0x22f5000) allocated [ CSSD]2013-05-23 17:40:26.535 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:40:26.535 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f5000 - 1,2,3 [ CSSD]2013-05-23 17:40:26.535 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f1f70) proc(0x22f5000) pid(3554/3554) version 11:1:1:4 [ CSSD]2013-05-23 17:40:26.551 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f5000), client(1/0x22ec830) [ CSSD]2013-05-23 17:40:26.551 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ec830) [ CSSD]2013-05-23 17:40:26.551 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:40:26.551 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:40:26.571 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:26.571 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:27.457 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:28.020 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f5000), client(2/0x22ece50) [ CSSD]2013-05-23 17:40:28.020 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22ece50) [ CSSD]2013-05-23 17:40:28.020 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:40:28.020 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 17:40:28.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:28.481 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:29.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:29.975 [1191196992] >TRACE: clssgmAllocProc: (0x2318ea0) allocated [ CSSD]2013-05-23 17:40:29.976 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:40:29.976 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318ea0 - 1,2,3 [ CSSD]2013-05-23 17:40:29.976 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318ea0) pid(3613) version 11:1:1:4 [ CSSD]2013-05-23 17:40:29.992 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2318ea0), client(1/0x2319400) [ CSSD]2013-05-23 17:40:29.992 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319400) [ CSSD]2013-05-23 17:40:29.992 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319400 with con 0x2319100, requested num -1 [ CSSD]2013-05-23 17:40:29.992 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:29.992 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 702 (0x7f431bfd4f10) [ CSSD]2013-05-23 17:40:29.992 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4f10 (RPC#702) tag(2be002d) sent to node 1 [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02be002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015389, gin 1, member 0 [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3510/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4f10 (RPC#702) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd4f10, ret 0, client 0x2319400 member 0x2318d80 [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319400 [ CSSD]2013-05-23 17:40:29.993 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 702 [ CSSD]2013-05-23 17:40:29.999 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319400) [ CSSD]2013-05-23 17:40:29.999 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319400), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:29.999 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:29.999 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 703 (0x7f431bfd4f90) [ CSSD]2013-05-23 17:40:29.999 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd4f90 (RPC#703) tag(2bf002d) sent to node 1 [ CSSD]2013-05-23 17:40:30.000 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02bf002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:30.000 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:30.000 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd4f90 (RPC#703) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:30.000 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd4f90, ret 0, client 0x2319400 member 0x2318d80 [ CSSD]2013-05-23 17:40:30.000 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 703 [ CSSD]2013-05-23 17:40:30.000 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:40:30.202 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318ea0 [ CSSD]2013-05-23 17:40:30.202 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318ea0) con(0x22ed550) skgpid ospid 3613 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:30.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:30.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:30.573 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:30.573 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:40:31.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:32.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:32.560 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:33.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:34.566 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:34.566 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:35.572 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:35.576 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:35.576 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:36.056 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:40:36.057 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 704 (0x7f431bfd5010) [ CSSD]2013-05-23 17:40:36.057 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5010 (RPC#704) tag(2c0002d) sent to node 1 [ CSSD]2013-05-23 17:40:36.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:36.057 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 704 (0x7f431bfd5010) [ CSSD]2013-05-23 17:40:36.057 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5010 (RPC#704) state 6, flags 0x202 [ CSSD]2013-05-23 17:40:36.057 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5010, ret 0, client (nil) [ CSSD]2013-05-23 17:40:36.057 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 704 [ CSSD]2013-05-23 17:40:36.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:36.178 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015390, gin 1, member 0 [ CSSD]2013-05-23 17:40:36.178 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:40:36.178 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:40:36.184 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:36.184 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:40:36.184 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:40:36.577 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:36.577 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:37.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:38.583 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:38.583 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:39.629 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:39.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:39.917 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015391, gin 1, member 0 [ CSSD]2013-05-23 17:40:39.917 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:40:39.917 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:40:39.924 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:39.924 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:40:39.924 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:40:40.066 [1191196992] >TRACE: clssgmAllocProc: (0x22ee120) allocated [ CSSD]2013-05-23 17:40:40.067 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:40:40.067 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee120 - 1,2,3 [ CSSD]2013-05-23 17:40:40.067 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22ee120) pid(3728) version 11:1:1:4 [ CSSD]2013-05-23 17:40:40.072 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ee120), client(1/0x23194e0) [ CSSD]2013-05-23 17:40:40.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:40:40.072 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23194e0 with con 0x2318fd0, requested num -1 [ CSSD]2013-05-23 17:40:40.072 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:40.072 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 705 (0x7f431bfd5090) [ CSSD]2013-05-23 17:40:40.073 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5090 (RPC#705) tag(2c1002d) sent to node 1 [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c1002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015392, gin 1, member 0 [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e33d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5090 (RPC#705) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5090, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23194e0 [ CSSD]2013-05-23 17:40:40.073 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 705 [ CSSD]2013-05-23 17:40:40.079 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:40:40.079 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23194e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:40.079 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:40.079 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 706 (0x7f431bfd5110) [ CSSD]2013-05-23 17:40:40.079 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5110 (RPC#706) tag(2c2002d) sent to node 1 [ CSSD]2013-05-23 17:40:40.079 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02c2002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:40.079 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:40.079 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5110 (RPC#706) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:40.079 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5110, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:40:40.080 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 706 [ CSSD]2013-05-23 17:40:40.080 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:40:40.282 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee120 [ CSSD]2013-05-23 17:40:40.282 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee120) con(0x22ec9a0) skgpid ospid 3728 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:40.579 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:40.579 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:40.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:40.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:41.654 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:42.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:42.610 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:43.666 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:44.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:44.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:45.233 [1191196992] >TRACE: clssgmAllocProc: (0x22ee120) allocated [ CSSD]2013-05-23 17:40:45.233 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:40:45.234 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee120 - 1,2,3 [ CSSD]2013-05-23 17:40:45.234 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22ee120) pid(3819) version 11:1:1:4 [ CSSD]2013-05-23 17:40:45.239 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ee120), client(1/0x23194e0) [ CSSD]2013-05-23 17:40:45.239 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:40:45.239 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23194e0 with con 0x2318fd0, requested num -1 [ CSSD]2013-05-23 17:40:45.239 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:45.239 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 707 (0x7f431bfd5190) [ CSSD]2013-05-23 17:40:45.239 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5190 (RPC#707) tag(2c3002d) sent to node 1 [ CSSD]2013-05-23 17:40:45.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c3002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:45.239 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015393, gin 1, member 0 [ CSSD]2013-05-23 17:40:45.239 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e33d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:40:45.240 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:40:45.240 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:40:45.240 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5190 (RPC#707) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:45.240 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5190, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:40:45.240 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23194e0 [ CSSD]2013-05-23 17:40:45.240 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 707 [ CSSD]2013-05-23 17:40:45.245 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:40:45.245 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23194e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:45.245 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:45.245 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 708 (0x7f431bfd5210) [ CSSD]2013-05-23 17:40:45.245 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5210 (RPC#708) tag(2c4002d) sent to node 1 [ CSSD]2013-05-23 17:40:45.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02c4002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:45.246 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:45.246 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5210 (RPC#708) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:45.246 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5210, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:40:45.246 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 708 [ CSSD]2013-05-23 17:40:45.246 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:40:45.448 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee120 [ CSSD]2013-05-23 17:40:45.448 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee120) con(0x22ec9a0) skgpid ospid 3819 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:45.581 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:45.581 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:45.672 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:46.081 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:40:46.082 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 709 (0x7f431bfd5290) [ CSSD]2013-05-23 17:40:46.082 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5290 (RPC#709) tag(2c5002d) sent to node 1 [ CSSD]2013-05-23 17:40:46.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:46.082 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 709 (0x7f431bfd5290) [ CSSD]2013-05-23 17:40:46.082 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5290 (RPC#709) state 6, flags 0x202 [ CSSD]2013-05-23 17:40:46.082 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5290, ret 0, client (nil) [ CSSD]2013-05-23 17:40:46.082 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 709 [ CSSD]2013-05-23 17:40:46.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:46.647 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:47.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:48.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:48.653 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:49.730 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:50.585 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:50.585 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:50.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:50.665 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:51.735 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:52.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:52.711 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:53.746 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:54.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:54.743 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:55.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:55.166 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015394, gin 1, member 0 [ CSSD]2013-05-23 17:40:55.166 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:40:55.166 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:40:55.173 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:55.173 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:40:55.173 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:40:55.588 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:40:55.588 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:40:55.752 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:56.106 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:40:56.106 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 710 (0x7f431bfd5310) [ CSSD]2013-05-23 17:40:56.106 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5310 (RPC#710) tag(2c6002d) sent to node 1 [ CSSD]2013-05-23 17:40:56.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:56.106 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 710 (0x7f431bfd5310) [ CSSD]2013-05-23 17:40:56.106 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5310 (RPC#710) state 6, flags 0x202 [ CSSD]2013-05-23 17:40:56.106 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5310, ret 0, client (nil) [ CSSD]2013-05-23 17:40:56.107 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 710 [ CSSD]2013-05-23 17:40:56.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:56.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:57.777 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:40:58.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:40:58.775 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:40:59.409 [1191196992] >TRACE: clssgmAllocProc: (0x22ee120) allocated [ CSSD]2013-05-23 17:40:59.410 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:40:59.411 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee120 - 1,2,3 [ CSSD]2013-05-23 17:40:59.411 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22ee120) pid(3984) version 11:1:1:4 [ CSSD]2013-05-23 17:40:59.417 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ee120), client(1/0x23194e0) [ CSSD]2013-05-23 17:40:59.417 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:40:59.417 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23194e0 with con 0x2318fd0, requested num -1 [ CSSD]2013-05-23 17:40:59.417 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:59.417 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 711 (0x7f431bfd5390) [ CSSD]2013-05-23 17:40:59.417 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5390 (RPC#711) tag(2c7002d) sent to node 1 [ CSSD]2013-05-23 17:40:59.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c7002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:59.417 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015395, gin 1, member 0 [ CSSD]2013-05-23 17:40:59.417 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e33d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:40:59.417 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:40:59.417 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:40:59.417 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5390 (RPC#711) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:59.418 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5390, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:40:59.418 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23194e0 [ CSSD]2013-05-23 17:40:59.418 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 711 [ CSSD]2013-05-23 17:40:59.423 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:40:59.423 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23194e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:59.423 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:40:59.423 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 712 (0x7f431bfd5410) [ CSSD]2013-05-23 17:40:59.423 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5410 (RPC#712) tag(2c8002d) sent to node 1 [ CSSD]2013-05-23 17:40:59.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02c8002d) incarnation(174631051) [ CSSD]2013-05-23 17:40:59.423 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:40:59.423 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5410 (RPC#712) state 6, flags 0x100 [ CSSD]2013-05-23 17:40:59.423 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5410, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:40:59.423 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 712 [ CSSD]2013-05-23 17:40:59.423 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:40:59.626 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee120 [ CSSD]2013-05-23 17:40:59.626 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee120) con(0x22ec9a0) skgpid ospid 3984 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:40:59.781 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:00.477 [1191196992] >TRACE: clssgmAllocProc: (0x2318ea0) allocated [ CSSD]2013-05-23 17:41:00.478 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:41:00.478 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318ea0 - 1,2,3 [ CSSD]2013-05-23 17:41:00.478 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318ea0) pid(3999) version 11:1:1:4 [ CSSD]2013-05-23 17:41:00.484 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2318ea0), client(1/0x2319400) [ CSSD]2013-05-23 17:41:00.484 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319400) [ CSSD]2013-05-23 17:41:00.484 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2319400 with con 0x2319100, requested num -1 [ CSSD]2013-05-23 17:41:00.484 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:00.484 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 713 (0x7f431bfd5490) [ CSSD]2013-05-23 17:41:00.485 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5490 (RPC#713) tag(2c9002d) sent to node 1 [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02c9002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015396, gin 1, member 0 [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3510/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5490 (RPC#713) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5490, ret 0, client 0x2319400 member 0x2318d80 [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319400 [ CSSD]2013-05-23 17:41:00.485 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 713 [ CSSD]2013-05-23 17:41:00.491 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319400) [ CSSD]2013-05-23 17:41:00.491 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319400), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:00.491 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:00.491 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 714 (0x7f431bfd5510) [ CSSD]2013-05-23 17:41:00.491 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5510 (RPC#714) tag(2ca002d) sent to node 1 [ CSSD]2013-05-23 17:41:00.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ca002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:00.491 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:00.491 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5510 (RPC#714) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:00.491 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5510, ret 0, client 0x2319400 member 0x2318d80 [ CSSD]2013-05-23 17:41:00.492 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 714 [ CSSD]2013-05-23 17:41:00.492 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:41:00.591 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:00.591 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:00.694 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318ea0 [ CSSD]2013-05-23 17:41:00.694 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318ea0) con(0x22ed550) skgpid ospid 3999 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:00.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:00.846 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:01.786 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:02.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:02.871 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:03.838 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:04.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:04.881 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:05.594 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:05.594 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:05.863 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:06.130 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:41:06.131 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 715 (0x7f431bfd5590) [ CSSD]2013-05-23 17:41:06.131 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5590 (RPC#715) tag(2cb002d) sent to node 1 [ CSSD]2013-05-23 17:41:06.131 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:06.131 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 715 (0x7f431bfd5590) [ CSSD]2013-05-23 17:41:06.131 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5590 (RPC#715) state 6, flags 0x202 [ CSSD]2013-05-23 17:41:06.131 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5590, ret 0, client (nil) [ CSSD]2013-05-23 17:41:06.132 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 715 [ CSSD]2013-05-23 17:41:06.888 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:06.888 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:07.869 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:08.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:08.893 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:09.881 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:10.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:10.418 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015397, gin 1, member 0 [ CSSD]2013-05-23 17:41:10.418 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:41:10.418 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:41:10.425 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:10.425 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:41:10.425 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:41:10.597 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:10.597 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:10.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:10.946 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:11.887 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:12.952 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:12.952 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:13.938 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:14.600 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:14.600 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:41:14.964 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:14.964 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:15.738 [1191196992] >TRACE: clssgmAllocProc: (0x22ee120) allocated [ CSSD]2013-05-23 17:41:15.739 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:41:15.739 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee120 - 1,2,3 [ CSSD]2013-05-23 17:41:15.739 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22ee120) pid(4155) version 11:1:1:4 [ CSSD]2013-05-23 17:41:15.745 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22ee120), client(1/0x23194e0) [ CSSD]2013-05-23 17:41:15.745 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:41:15.745 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23194e0 with con 0x2318fd0, requested num -1 [ CSSD]2013-05-23 17:41:15.745 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:15.745 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 716 (0x7f431bfd5610) [ CSSD]2013-05-23 17:41:15.745 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5610 (RPC#716) tag(2cc002d) sent to node 1 [ CSSD]2013-05-23 17:41:15.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02cc002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:15.745 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015398, gin 1, member 0 [ CSSD]2013-05-23 17:41:15.745 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e33d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:41:15.746 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:41:15.746 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:41:15.746 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5610 (RPC#716) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:15.746 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5610, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:41:15.746 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23194e0 [ CSSD]2013-05-23 17:41:15.746 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 716 [ CSSD]2013-05-23 17:41:15.751 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23194e0) [ CSSD]2013-05-23 17:41:15.752 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23194e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:15.752 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:15.752 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 717 (0x7f431bfd5690) [ CSSD]2013-05-23 17:41:15.752 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5690 (RPC#717) tag(2cd002d) sent to node 1 [ CSSD]2013-05-23 17:41:15.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02cd002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:15.752 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:15.752 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5690 (RPC#717) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:15.752 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5690, ret 0, client 0x23194e0 member 0x2318d80 [ CSSD]2013-05-23 17:41:15.752 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 717 [ CSSD]2013-05-23 17:41:15.752 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:41:15.944 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:15.954 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee120 [ CSSD]2013-05-23 17:41:15.954 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee120) con(0x22ec9a0) skgpid ospid 4155 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:16.155 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:41:16.155 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 718 (0x7f431bfd5710) [ CSSD]2013-05-23 17:41:16.155 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5710 (RPC#718) tag(2ce002d) sent to node 1 [ CSSD]2013-05-23 17:41:16.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:16.156 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 718 (0x7f431bfd5710) [ CSSD]2013-05-23 17:41:16.156 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5710 (RPC#718) state 6, flags 0x202 [ CSSD]2013-05-23 17:41:16.156 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5710, ret 0, client (nil) [ CSSD]2013-05-23 17:41:16.156 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 718 [ CSSD]2013-05-23 17:41:16.990 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:16.990 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:17.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:18.996 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:18.996 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:19.603 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:19.603 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:20.002 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:21.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:21.047 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:22.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:22.085 [1191196992] >TRACE: clssgmAllocProc: (0x2318ea0) allocated [ CSSD]2013-05-23 17:41:22.086 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:41:22.087 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318ea0 - 1,2,3 [ CSSD]2013-05-23 17:41:22.087 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318ea0) pid(4254) version 11:1:1:4 [ CSSD]2013-05-23 17:41:22.092 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2318ea0), client(1/0x2319400) [ CSSD]2013-05-23 17:41:22.092 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2319400) [ CSSD]2013-05-23 17:41:22.092 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x2319400 with con 0x2319100, requested num -1 [ CSSD]2013-05-23 17:41:22.092 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:41:22.092 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 719 (0x7f431bfd5790) [ CSSD]2013-05-23 17:41:22.092 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5790 (RPC#719) tag(2cf002d) sent to node 1 [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02cf002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015399, gin 1, member 0 [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3510/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5790 (RPC#719) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5790, ret 0, client 0x2319400 member 0x2318d80 [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2319400 [ CSSD]2013-05-23 17:41:22.093 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 719 [ CSSD]2013-05-23 17:41:22.099 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2319400) [ CSSD]2013-05-23 17:41:22.099 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2319400), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:41:22.099 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:41:22.099 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 720 (0x7f431bfd5810) [ CSSD]2013-05-23 17:41:22.100 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5810 (RPC#720) tag(2d0002d) sent to node 1 [ CSSD]2013-05-23 17:41:22.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d0002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:22.100 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:41:22.100 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5810 (RPC#720) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:22.100 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5810, ret 0, client 0x2319400 member 0x2318d80 [ CSSD]2013-05-23 17:41:22.100 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 720 [ CSSD]2013-05-23 17:41:22.100 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:41:22.207 [1191196992] >TRACE: clssgmAllocProc: (0x22e5380) allocated [ CSSD]2013-05-23 17:41:22.208 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:41:22.209 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e5380 - 1,2,3 [ CSSD]2013-05-23 17:41:22.209 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22e5380) pid(4261) version 11:1:1:4 [ CSSD]2013-05-23 17:41:22.223 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:41:22.225 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:41:22.227 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e5380 [ CSSD]2013-05-23 17:41:22.227 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e5380) con(0x22ec9a0) skgpid ospid 4261 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:22.302 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318ea0 [ CSSD]2013-05-23 17:41:22.302 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318ea0) con(0x22ed550) skgpid ospid 4254 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:23.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:23.052 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:24.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:24.606 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:24.606 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:25.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:25.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:25.138 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:41:25.138 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:41:25.138 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:41:25.138 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:41:25.139 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a490 [ CSSD]2013-05-23 17:41:25.139 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a490) con(0x7f431c134df0) skgpid 31346 ospid 31346 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:25.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:25.678 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015400, gin 1, member 0 [ CSSD]2013-05-23 17:41:25.678 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:41:25.678 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:41:25.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:25.685 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:41:25.685 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:41:26.044 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:26.193 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:41:26.193 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 721 (0x7f431bfd5890) [ CSSD]2013-05-23 17:41:26.194 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5890 (RPC#721) tag(2d1002d) sent to node 1 [ CSSD]2013-05-23 17:41:26.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:26.194 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 721 (0x7f431bfd5890) [ CSSD]2013-05-23 17:41:26.194 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5890 (RPC#721) state 6, flags 0x202 [ CSSD]2013-05-23 17:41:26.194 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5890, ret 0, client (nil) [ CSSD]2013-05-23 17:41:26.194 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 721 [ CSSD]2013-05-23 17:41:27.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:27.069 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:28.050 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:29.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:29.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:29.609 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:29.609 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:30.061 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:30.987 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:41:30.988 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:41:30.989 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:41:30.989 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(4352) version 11:1:1:4 [ CSSD]2013-05-23 17:41:30.994 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x23193f0) [ CSSD]2013-05-23 17:41:30.994 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:41:30.994 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23193f0 with con 0x23190f0, requested num -1 [ CSSD]2013-05-23 17:41:30.994 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:30.994 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 722 (0x7f431bfd5910) [ CSSD]2013-05-23 17:41:30.995 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5910 (RPC#722) tag(2d2002d) sent to node 1 [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02d2002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015401, gin 1, member 0 [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5910 (RPC#722) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5910, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23193f0 [ CSSD]2013-05-23 17:41:30.995 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 722 [ CSSD]2013-05-23 17:41:31.003 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:41:31.003 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23193f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:31.003 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:31.003 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 723 (0x7f431bfd5990) [ CSSD]2013-05-23 17:41:31.003 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5990 (RPC#723) tag(2d3002d) sent to node 1 [ CSSD]2013-05-23 17:41:31.004 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d3002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:31.004 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:31.004 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5990 (RPC#723) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:31.004 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5990, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:41:31.004 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 723 [ CSSD]2013-05-23 17:41:31.004 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:41:31.087 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:31.087 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:31.207 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:41:31.207 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 4352 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:32.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:33.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:33.113 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:34.079 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:34.612 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:34.612 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:35.145 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:35.145 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:36.104 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:36.219 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:41:36.219 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 724 (0x7f431bfd5a10) [ CSSD]2013-05-23 17:41:36.219 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5a10 (RPC#724) tag(2d4002d) sent to node 1 [ CSSD]2013-05-23 17:41:36.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:36.220 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 724 (0x7f431bfd5a10) [ CSSD]2013-05-23 17:41:36.220 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5a10 (RPC#724) state 6, flags 0x202 [ CSSD]2013-05-23 17:41:36.220 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5a10, ret 0, client (nil) [ CSSD]2013-05-23 17:41:36.220 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 724 [ CSSD]2013-05-23 17:41:36.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:36.434 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015402, gin 1, member 0 [ CSSD]2013-05-23 17:41:36.434 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:41:36.435 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:41:36.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:36.441 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:41:36.441 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:41:37.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:37.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:38.109 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:39.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:39.216 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:39.614 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:39.615 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:40.142 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:40.329 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:41:40.330 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:41:40.330 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:41:40.330 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(4458) version 11:1:1:4 [ CSSD]2013-05-23 17:41:40.336 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x23193f0) [ CSSD]2013-05-23 17:41:40.336 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:41:40.336 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23193f0 with con 0x23190f0, requested num -1 [ CSSD]2013-05-23 17:41:40.336 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:40.336 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 725 (0x7f431bfd5a90) [ CSSD]2013-05-23 17:41:40.336 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5a90 (RPC#725) tag(2d5002d) sent to node 1 [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02d5002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015403, gin 1, member 0 [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5a90 (RPC#725) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5a90, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:41:40.336 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23193f0 [ CSSD]2013-05-23 17:41:40.337 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 725 [ CSSD]2013-05-23 17:41:40.342 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:41:40.342 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23193f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:40.342 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:40.342 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 726 (0x7f431bfd5b10) [ CSSD]2013-05-23 17:41:40.342 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5b10 (RPC#726) tag(2d6002d) sent to node 1 [ CSSD]2013-05-23 17:41:40.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d6002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:40.343 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:40.343 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5b10 (RPC#726) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:40.343 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5b10, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:41:40.343 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 726 [ CSSD]2013-05-23 17:41:40.343 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:41:40.545 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:41:40.545 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 4458 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:40.933 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:40.933 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015404, gin 1, member 0 [ CSSD]2013-05-23 17:41:40.933 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:41:40.933 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:41:40.940 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:40.940 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:41:40.940 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:41:41.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:41.248 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:42.148 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:43.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:43.253 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:44.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:44.617 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:44.617 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:45.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:45.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:46.165 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:46.237 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:41:46.237 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:41:46.238 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:41:46.238 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(4527) version 11:1:1:4 [ CSSD]2013-05-23 17:41:46.244 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x23193f0) [ CSSD]2013-05-23 17:41:46.244 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:41:46.244 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23193f0 with con 0x23190f0, requested num -1 [ CSSD]2013-05-23 17:41:46.244 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:46.244 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 727 (0x7f431bfd5b90) [ CSSD]2013-05-23 17:41:46.244 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5b90 (RPC#727) tag(2d7002d) sent to node 1 [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02d7002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015405, gin 1, member 0 [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5b90 (RPC#727) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5b90, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:41:46.245 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23193f0 [ CSSD]2013-05-23 17:41:46.246 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 727 [ CSSD]2013-05-23 17:41:46.247 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:41:46.247 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 728 (0x7f431bfd5c10) [ CSSD]2013-05-23 17:41:46.247 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5c10 (RPC#728) tag(2d8002d) sent to node 1 [ CSSD]2013-05-23 17:41:46.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:46.248 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 728 (0x7f431bfd5c10) [ CSSD]2013-05-23 17:41:46.248 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5c10 (RPC#728) state 6, flags 0x202 [ CSSD]2013-05-23 17:41:46.248 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5c10, ret 0, client (nil) [ CSSD]2013-05-23 17:41:46.248 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 728 [ CSSD]2013-05-23 17:41:46.252 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:41:46.252 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23193f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:46.252 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:41:46.252 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 729 (0x7f431bfd5c90) [ CSSD]2013-05-23 17:41:46.252 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5c90 (RPC#729) tag(2d9002d) sent to node 1 [ CSSD]2013-05-23 17:41:46.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02d9002d) incarnation(174631051) [ CSSD]2013-05-23 17:41:46.253 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:41:46.253 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5c90 (RPC#729) state 6, flags 0x100 [ CSSD]2013-05-23 17:41:46.253 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5c90, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:41:46.253 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 729 [ CSSD]2013-05-23 17:41:46.253 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:41:46.455 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:41:46.455 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 4527 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:41:47.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:47.371 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:48.237 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:49.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:49.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:49.620 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:49.620 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:50.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:51.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:51.388 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:52.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:53.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:53.394 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:54.280 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:54.623 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:54.623 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:41:55.446 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:55.446 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:56.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:56.183 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015406, gin 1, member 0 [ CSSD]2013-05-23 17:41:56.183 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:41:56.183 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:41:56.189 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:56.189 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:41:56.189 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:41:56.271 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:41:56.271 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 730 (0x7f431bfd5d10) [ CSSD]2013-05-23 17:41:56.271 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5d10 (RPC#730) tag(2da002d) sent to node 1 [ CSSD]2013-05-23 17:41:56.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:56.272 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 730 (0x7f431bfd5d10) [ CSSD]2013-05-23 17:41:56.272 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5d10 (RPC#730) state 6, flags 0x202 [ CSSD]2013-05-23 17:41:56.272 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5d10, ret 0, client (nil) [ CSSD]2013-05-23 17:41:56.272 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 730 [ CSSD]2013-05-23 17:41:56.285 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:57.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:57.472 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:58.338 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:41:59.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:41:59.478 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:41:59.626 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:41:59.626 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:00.344 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:01.487 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:42:01.488 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:42:01.488 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:42:01.488 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(4666) version 11:1:1:4 [ CSSD]2013-05-23 17:42:01.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:01.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:01.504 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x22e3300) [ CSSD]2013-05-23 17:42:01.504 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e3300) [ CSSD]2013-05-23 17:42:01.504 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e3300 with con 0x22ec9a0, requested num -1 [ CSSD]2013-05-23 17:42:01.504 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:01.504 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 731 (0x7f431bfd5d90) [ CSSD]2013-05-23 17:42:01.504 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5d90 (RPC#731) tag(2db002d) sent to node 1 [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02db002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015407, gin 1, member 0 [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3470/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5d90 (RPC#731) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5d90, ret 0, client 0x22e3300 member 0x2318d80 [ CSSD]2013-05-23 17:42:01.504 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e3300 [ CSSD]2013-05-23 17:42:01.505 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 731 [ CSSD]2013-05-23 17:42:01.511 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e3300) [ CSSD]2013-05-23 17:42:01.511 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e3300), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:01.511 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:01.511 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 732 (0x7f431bfd5e10) [ CSSD]2013-05-23 17:42:01.511 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5e10 (RPC#732) tag(2dc002d) sent to node 1 [ CSSD]2013-05-23 17:42:01.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02dc002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:01.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:01.511 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5e10 (RPC#732) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:01.511 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5e10, ret 0, client 0x22e3300 member 0x2318d80 [ CSSD]2013-05-23 17:42:01.512 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 732 [ CSSD]2013-05-23 17:42:01.512 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:01.714 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:42:01.714 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 4666 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:02.370 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:03.496 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:03.496 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:04.400 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:04.629 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:04.629 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:05.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:05.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:06.295 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:42:06.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 733 (0x7f431bfd5e90) [ CSSD]2013-05-23 17:42:06.296 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5e90 (RPC#733) tag(2dd002d) sent to node 1 [ CSSD]2013-05-23 17:42:06.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:06.296 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 733 (0x7f431bfd5e90) [ CSSD]2013-05-23 17:42:06.296 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5e90 (RPC#733) state 6, flags 0x202 [ CSSD]2013-05-23 17:42:06.296 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd5e90, ret 0, client (nil) [ CSSD]2013-05-23 17:42:06.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 733 [ CSSD]2013-05-23 17:42:06.405 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:07.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:07.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:08.438 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:09.456 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:42:09.457 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:42:09.458 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:42:09.458 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(4763) version 11:1:1:4 [ CSSD]2013-05-23 17:42:09.464 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x23193f0) [ CSSD]2013-05-23 17:42:09.464 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:42:09.464 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23193f0 with con 0x23190f0, requested num -1 [ CSSD]2013-05-23 17:42:09.464 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:09.464 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 734 (0x7f431bfd5f10) [ CSSD]2013-05-23 17:42:09.464 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5f10 (RPC#734) tag(2de002d) sent to node 1 [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02de002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015408, gin 1, member 0 [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5f10 (RPC#734) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd5f10, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23193f0 [ CSSD]2013-05-23 17:42:09.465 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 734 [ CSSD]2013-05-23 17:42:09.472 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:42:09.472 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23193f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:09.472 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:09.472 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 735 (0x7f431bfd5f90) [ CSSD]2013-05-23 17:42:09.472 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd5f90 (RPC#735) tag(2df002d) sent to node 1 [ CSSD]2013-05-23 17:42:09.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02df002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:09.472 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:09.472 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd5f90 (RPC#735) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:09.472 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd5f90, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:42:09.472 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 735 [ CSSD]2013-05-23 17:42:09.472 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:09.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:09.529 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:09.632 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:09.632 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:09.675 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:42:09.675 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 4763 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:10.444 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:11.440 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:11.440 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015409, gin 1, member 0 [ CSSD]2013-05-23 17:42:11.440 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:42:11.440 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:42:11.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:11.445 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:42:11.445 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:42:11.559 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:11.559 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:12.450 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:13.565 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:13.565 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:14.461 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:14.635 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:14.635 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:15.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:15.576 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:16.321 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:42:16.321 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 736 (0x7f431bfd6010) [ CSSD]2013-05-23 17:42:16.321 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6010 (RPC#736) tag(2e0002d) sent to node 1 [ CSSD]2013-05-23 17:42:16.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:16.322 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 736 (0x7f431bfd6010) [ CSSD]2013-05-23 17:42:16.322 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6010 (RPC#736) state 6, flags 0x202 [ CSSD]2013-05-23 17:42:16.322 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6010, ret 0, client (nil) [ CSSD]2013-05-23 17:42:16.322 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 736 [ CSSD]2013-05-23 17:42:16.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:16.756 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:42:16.757 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:42:16.757 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:42:16.757 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(5031) version 11:1:1:4 [ CSSD]2013-05-23 17:42:16.762 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x23193f0) [ CSSD]2013-05-23 17:42:16.762 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:42:16.762 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23193f0 with con 0x23190f0, requested num -1 [ CSSD]2013-05-23 17:42:16.762 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:16.762 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 737 (0x7f431bfd6090) [ CSSD]2013-05-23 17:42:16.763 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6090 (RPC#737) tag(2e1002d) sent to node 1 [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e1002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015410, gin 1, member 0 [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6090 (RPC#737) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6090, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23193f0 [ CSSD]2013-05-23 17:42:16.763 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 737 [ CSSD]2013-05-23 17:42:16.769 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:42:16.769 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23193f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:16.769 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:16.769 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 738 (0x7f431bfd6110) [ CSSD]2013-05-23 17:42:16.769 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6110 (RPC#738) tag(2e2002d) sent to node 1 [ CSSD]2013-05-23 17:42:16.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e2002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:16.770 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:16.770 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6110 (RPC#738) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:16.770 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6110, ret 0, client 0x23193f0 member 0x2318d80 [ CSSD]2013-05-23 17:42:16.770 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 738 [ CSSD]2013-05-23 17:42:16.770 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:16.972 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:42:16.972 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 5031 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:17.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:17.601 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:18.479 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:19.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:19.607 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:19.638 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:19.639 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:20.485 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:20.864 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:42:20.865 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:42:20.865 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:42:20.865 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ed550) proc(0x2318e90) pid(5103) version 11:1:1:4 [ CSSD]2013-05-23 17:42:20.998 [1191196992] >TRACE: clssgmAllocProc: (0x2321190) allocated [ CSSD]2013-05-23 17:42:20.999 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:42:21.000 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2321190 - 1,2,3 [ CSSD]2013-05-23 17:42:21.000 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e33e0) proc(0x2321190) pid(5113) version 11:1:1:4 [ CSSD]2013-05-23 17:42:21.016 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:42:21.018 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:42:21.020 [1191196992] >TRACE: clssgmDeadProc: proc 0x2321190 [ CSSD]2013-05-23 17:42:21.020 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2321190) con(0x22e33e0) skgpid ospid 5113 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:21.096 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(83/0x23192f0) [ CSSD]2013-05-23 17:42:21.096 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 83 (0x23192f0) [ CSSD]2013-05-23 17:42:21.096 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x23192f0 with con 0x22e3300, requested num -1 [ CSSD]2013-05-23 17:42:21.096 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:42:21.096 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 739 (0x7f431bfd6190) [ CSSD]2013-05-23 17:42:21.097 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6190 (RPC#739) tag(2e3002d) sent to node 1 [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e3002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015411, gin 1, member 0 [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x23211d0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6190 (RPC#739) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6190, ret 0, client 0x23192f0 member 0x2318c90 [ CSSD]2013-05-23 17:42:21.097 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23192f0 [ CSSD]2013-05-23 17:42:21.098 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 739 [ CSSD]2013-05-23 17:42:21.103 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 83 (0x23192f0) [ CSSD]2013-05-23 17:42:21.103 [1191196992] >TRACE: clssgmExitGrock: client 83 (0x23192f0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:42:21.103 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318c90) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:42:21.103 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 740 (0x7f431bfd6210) [ CSSD]2013-05-23 17:42:21.103 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6210 (RPC#740) tag(2e4002d) sent to node 1 [ CSSD]2013-05-23 17:42:21.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e4002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:21.104 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:42:21.104 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6210 (RPC#740) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:21.104 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6210, ret 0, client 0x23192f0 member 0x2318c90 [ CSSD]2013-05-23 17:42:21.104 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 740 [ CSSD]2013-05-23 17:42:21.104 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x2318c90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:21.105 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:42:21.105 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ed550) skgpid ospid 5103 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:21.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:21.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:22.491 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:23.640 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:23.641 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:42:23.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:23.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:24.511 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:25.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:25.677 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:26.351 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:42:26.351 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 741 (0x7f431bfd6290) [ CSSD]2013-05-23 17:42:26.351 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6290 (RPC#741) tag(2e5002d) sent to node 1 [ CSSD]2013-05-23 17:42:26.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:26.352 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 741 (0x7f431bfd6290) [ CSSD]2013-05-23 17:42:26.352 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6290 (RPC#741) state 6, flags 0x202 [ CSSD]2013-05-23 17:42:26.352 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6290, ret 0, client (nil) [ CSSD]2013-05-23 17:42:26.352 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 741 [ CSSD]2013-05-23 17:42:26.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:26.697 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:26.697 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015412, gin 1, member 0 [ CSSD]2013-05-23 17:42:26.697 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:42:26.697 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:42:26.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:26.704 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:42:26.704 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:42:27.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:27.683 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:28.549 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:28.643 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:28.643 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:29.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:29.755 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:30.555 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:31.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:31.781 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:32.009 [1191196992] >TRACE: clssgmAllocProc: (0x22ee120) allocated [ CSSD]2013-05-23 17:42:32.010 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:42:32.010 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee120 - 1,2,3 [ CSSD]2013-05-23 17:42:32.010 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22ee120) pid(5465) version 11:1:1:4 [ CSSD]2013-05-23 17:42:32.016 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee120), client(1/0x23194c0) [ CSSD]2013-05-23 17:42:32.016 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23194c0) [ CSSD]2013-05-23 17:42:32.016 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23194c0 with con 0x2318fc0, requested num -1 [ CSSD]2013-05-23 17:42:32.016 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:32.016 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 742 (0x7f431bfd6310) [ CSSD]2013-05-23 17:42:32.016 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6310 (RPC#742) tag(2e6002d) sent to node 1 [ CSSD]2013-05-23 17:42:32.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e6002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:32.016 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015413, gin 1, member 0 [ CSSD]2013-05-23 17:42:32.016 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e33d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:42:32.017 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:32.017 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:42:32.017 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6310 (RPC#742) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:32.017 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6310, ret 0, client 0x23194c0 member 0x2318d80 [ CSSD]2013-05-23 17:42:32.017 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23194c0 [ CSSD]2013-05-23 17:42:32.017 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 742 [ CSSD]2013-05-23 17:42:32.023 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23194c0) [ CSSD]2013-05-23 17:42:32.023 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23194c0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:32.023 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:32.023 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 743 (0x7f431bfd6390) [ CSSD]2013-05-23 17:42:32.023 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6390 (RPC#743) tag(2e7002d) sent to node 1 [ CSSD]2013-05-23 17:42:32.023 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02e7002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:32.023 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:32.023 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6390 (RPC#743) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:32.023 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6390, ret 0, client 0x23194c0 member 0x2318d80 [ CSSD]2013-05-23 17:42:32.023 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 743 [ CSSD]2013-05-23 17:42:32.023 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:32.226 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee120 [ CSSD]2013-05-23 17:42:32.226 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee120) con(0x22ec9a0) skgpid ospid 5465 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:32.561 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:33.646 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:33.647 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:33.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:33.786 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:34.572 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:35.797 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:35.797 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:36.375 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:42:36.375 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 744 (0x7f431bfd6410) [ CSSD]2013-05-23 17:42:36.375 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6410 (RPC#744) tag(2e8002d) sent to node 1 [ CSSD]2013-05-23 17:42:36.376 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:36.376 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 744 (0x7f431bfd6410) [ CSSD]2013-05-23 17:42:36.376 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6410 (RPC#744) state 6, flags 0x202 [ CSSD]2013-05-23 17:42:36.376 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6410, ret 0, client (nil) [ CSSD]2013-05-23 17:42:36.376 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 744 [ CSSD]2013-05-23 17:42:36.578 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:36.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:36.687 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015414, gin 1, member 0 [ CSSD]2013-05-23 17:42:36.687 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:42:36.687 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:42:36.692 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:36.693 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:42:36.693 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:42:37.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:37.803 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:38.589 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:38.650 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:38.650 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:39.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:39.856 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:40.590 [1191196992] >TRACE: clssgmAllocProc: (0x22ee120) allocated [ CSSD]2013-05-23 17:42:40.591 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:42:40.591 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ee120 - 1,2,3 [ CSSD]2013-05-23 17:42:40.591 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x22ee120) pid(5564) version 11:1:1:4 [ CSSD]2013-05-23 17:42:40.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:40.597 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ee120), client(1/0x23193f0) [ CSSD]2013-05-23 17:42:40.597 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:42:40.597 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23193f0 with con 0x23190f0, requested num -1 [ CSSD]2013-05-23 17:42:40.597 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:40.597 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 745 (0x7f431bfd6490) [ CSSD]2013-05-23 17:42:40.597 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6490 (RPC#745) tag(2e9002d) sent to node 1 [ CSSD]2013-05-23 17:42:40.597 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02e9002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:40.597 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015415, gin 1, member 0 [ CSSD]2013-05-23 17:42:40.597 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22e3500/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:42:40.598 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:40.598 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:42:40.598 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6490 (RPC#745) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:40.598 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6490, ret 0, client 0x23193f0 member 0x2318d60 [ CSSD]2013-05-23 17:42:40.598 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23193f0 [ CSSD]2013-05-23 17:42:40.598 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 745 [ CSSD]2013-05-23 17:42:40.605 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23193f0) [ CSSD]2013-05-23 17:42:40.605 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23193f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:40.605 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2318d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:40.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 746 (0x7f431bfd6510) [ CSSD]2013-05-23 17:42:40.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6510 (RPC#746) tag(2ea002d) sent to node 1 [ CSSD]2013-05-23 17:42:40.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ea002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:40.605 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:40.605 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6510 (RPC#746) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:40.605 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6510, ret 0, client 0x23193f0 member 0x2318d60 [ CSSD]2013-05-23 17:42:40.606 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 746 [ CSSD]2013-05-23 17:42:40.606 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2318d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:40.808 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ee120 [ CSSD]2013-05-23 17:42:40.808 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ee120) con(0x22ec9a0) skgpid ospid 5564 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:41.014 [1191196992] >TRACE: clssgmAllocProc: (0x2318e90) allocated [ CSSD]2013-05-23 17:42:41.015 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:42:41.015 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2318e90 - 1,2,3 [ CSSD]2013-05-23 17:42:41.015 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22ec9a0) proc(0x2318e90) pid(4296/4296) version 11:1:1:4 [ CSSD]2013-05-23 17:42:41.021 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(1/0x23193a0) [ CSSD]2013-05-23 17:42:41.021 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23193a0) [ CSSD]2013-05-23 17:42:41.021 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:42:41.021 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:42:41.880 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:41.880 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:41.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:41.956 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015416, gin 1, member 0 [ CSSD]2013-05-23 17:42:41.956 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:42:41.956 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:42:41.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:41.963 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:42:41.963 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:42:42.600 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:42.652 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:42.652 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:42:43.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:43.885 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:44.653 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:45.899 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:45.899 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:46.399 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:42:46.399 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 747 (0x7f431bfd6590) [ CSSD]2013-05-23 17:42:46.399 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6590 (RPC#747) tag(2eb002d) sent to node 1 [ CSSD]2013-05-23 17:42:46.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:46.400 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 747 (0x7f431bfd6590) [ CSSD]2013-05-23 17:42:46.400 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6590 (RPC#747) state 6, flags 0x202 [ CSSD]2013-05-23 17:42:46.400 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6590, ret 0, client (nil) [ CSSD]2013-05-23 17:42:46.400 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 747 [ CSSD]2013-05-23 17:42:46.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:47.259 [1191196992] >TRACE: clssgmAllocProc: (0x22e51c0) allocated [ CSSD]2013-05-23 17:42:47.260 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:42:47.261 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e51c0 - 1,2,3 [ CSSD]2013-05-23 17:42:47.261 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e3890) proc(0x22e51c0) pid(5676) version 11:1:1:4 [ CSSD]2013-05-23 17:42:47.266 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e51c0), client(1/0x2321d30) [ CSSD]2013-05-23 17:42:47.266 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321d30) [ CSSD]2013-05-23 17:42:47.266 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321d30 with con 0x22e5440, requested num -1 [ CSSD]2013-05-23 17:42:47.266 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:47.266 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 748 (0x7f431bfd6610) [ CSSD]2013-05-23 17:42:47.267 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6610 (RPC#748) tag(2ec002d) sent to node 1 [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ec002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015417, gin 1, member 0 [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321ea0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6610 (RPC#748) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6610, ret 0, client 0x2321d30 member 0x22e50a0 [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321d30 [ CSSD]2013-05-23 17:42:47.267 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 748 [ CSSD]2013-05-23 17:42:47.273 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321d30) [ CSSD]2013-05-23 17:42:47.273 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:47.273 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e50a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:42:47.273 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 749 (0x7f431bfd6690) [ CSSD]2013-05-23 17:42:47.273 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6690 (RPC#749) tag(2ed002d) sent to node 1 [ CSSD]2013-05-23 17:42:47.274 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02ed002d) incarnation(174631051) [ CSSD]2013-05-23 17:42:47.274 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:42:47.274 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6690 (RPC#749) state 6, flags 0x100 [ CSSD]2013-05-23 17:42:47.274 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6690, ret 0, client 0x2321d30 member 0x22e50a0 [ CSSD]2013-05-23 17:42:47.274 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 749 [ CSSD]2013-05-23 17:42:47.274 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e50a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:42:47.477 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e51c0 [ CSSD]2013-05-23 17:42:47.477 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e51c0) con(0x22e3890) skgpid ospid 5676 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:42:47.655 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:47.655 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:47.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:47.905 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:48.690 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:49.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:49.956 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:50.695 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:51.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:51.962 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:52.658 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:52.658 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:52.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:53.987 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:53.987 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:54.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:55.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:55.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:56.431 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:42:56.431 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 750 (0x7f431bfd6710) [ CSSD]2013-05-23 17:42:56.431 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6710 (RPC#750) tag(2ee002d) sent to node 1 [ CSSD]2013-05-23 17:42:56.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:56.432 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 750 (0x7f431bfd6710) [ CSSD]2013-05-23 17:42:56.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6710 (RPC#750) state 6, flags 0x202 [ CSSD]2013-05-23 17:42:56.432 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6710, ret 0, client (nil) [ CSSD]2013-05-23 17:42:56.432 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 750 [ CSSD]2013-05-23 17:42:56.799 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:42:57.208 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:57.208 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015418, gin 1, member 0 [ CSSD]2013-05-23 17:42:57.208 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:42:57.208 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:42:57.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:57.215 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:42:57.215 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:42:57.661 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:42:57.661 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:42:58.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:42:58.003 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:42:58.851 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:00.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:00.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:00.857 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:02.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:02.063 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:02.509 [1191196992] >TRACE: clssgmAllocProc: (0x22e50f0) allocated [ CSSD]2013-05-23 17:43:02.510 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:43:02.510 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e50f0 - 1,2,3 [ CSSD]2013-05-23 17:43:02.510 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e37c0) proc(0x22e50f0) pid(5835) version 11:1:1:4 [ CSSD]2013-05-23 17:43:02.516 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e50f0), client(1/0x2321c60) [ CSSD]2013-05-23 17:43:02.516 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321c60) [ CSSD]2013-05-23 17:43:02.516 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321c60 with con 0x22e4ee0, requested num -1 [ CSSD]2013-05-23 17:43:02.516 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:02.516 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 751 (0x7f431bfd6790) [ CSSD]2013-05-23 17:43:02.516 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6790 (RPC#751) tag(2ef002d) sent to node 1 [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ef002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015419, gin 1, member 0 [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321dd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6790 (RPC#751) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6790, ret 0, client 0x2321c60 member 0x23220d0 [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321c60 [ CSSD]2013-05-23 17:43:02.517 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 751 [ CSSD]2013-05-23 17:43:02.523 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321c60) [ CSSD]2013-05-23 17:43:02.523 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321c60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:02.523 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23220d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:02.523 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 752 (0x7f431bfd6810) [ CSSD]2013-05-23 17:43:02.523 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6810 (RPC#752) tag(2f0002d) sent to node 1 [ CSSD]2013-05-23 17:43:02.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02f0002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:02.523 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:02.523 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6810 (RPC#752) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:02.523 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6810, ret 0, client 0x2321c60 member 0x23220d0 [ CSSD]2013-05-23 17:43:02.523 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 752 [ CSSD]2013-05-23 17:43:02.523 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23220d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:43:02.664 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:02.664 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:02.726 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e50f0 [ CSSD]2013-05-23 17:43:02.726 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e50f0) con(0x22e37c0) skgpid ospid 5835 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:43:02.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:04.089 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:04.089 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:04.873 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:06.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:06.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:06.463 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:43:06.464 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 753 (0x7f431bfd6890) [ CSSD]2013-05-23 17:43:06.464 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6890 (RPC#753) tag(2f1002d) sent to node 1 [ CSSD]2013-05-23 17:43:06.464 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:06.464 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 753 (0x7f431bfd6890) [ CSSD]2013-05-23 17:43:06.464 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6890 (RPC#753) state 6, flags 0x202 [ CSSD]2013-05-23 17:43:06.464 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6890, ret 0, client (nil) [ CSSD]2013-05-23 17:43:06.464 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 753 [ CSSD]2013-05-23 17:43:06.667 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:06.667 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:43:06.899 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:08.124 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:08.124 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:08.951 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:10.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:10.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:10.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:11.670 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:11.670 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:12.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:12.164 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:12.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:12.458 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015420, gin 1, member 0 [ CSSD]2013-05-23 17:43:12.458 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:43:12.458 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:43:12.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:12.465 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:43:12.465 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:43:12.981 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:14.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:14.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:15.014 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:16.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:16.220 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:16.490 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:43:16.490 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 754 (0x7f431bfd6910) [ CSSD]2013-05-23 17:43:16.490 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6910 (RPC#754) tag(2f2002d) sent to node 1 [ CSSD]2013-05-23 17:43:16.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:16.491 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 754 (0x7f431bfd6910) [ CSSD]2013-05-23 17:43:16.491 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6910 (RPC#754) state 6, flags 0x202 [ CSSD]2013-05-23 17:43:16.491 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6910, ret 0, client (nil) [ CSSD]2013-05-23 17:43:16.491 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 754 [ CSSD]2013-05-23 17:43:16.673 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:16.673 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:17.019 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:17.757 [1191196992] >TRACE: clssgmAllocProc: (0x22e51c0) allocated [ CSSD]2013-05-23 17:43:17.758 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:43:17.758 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e51c0 - 1,2,3 [ CSSD]2013-05-23 17:43:17.759 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e3890) proc(0x22e51c0) pid(6022) version 11:1:1:4 [ CSSD]2013-05-23 17:43:17.764 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e51c0), client(1/0x2321d30) [ CSSD]2013-05-23 17:43:17.764 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321d30) [ CSSD]2013-05-23 17:43:17.764 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321d30 with con 0x22e5440, requested num -1 [ CSSD]2013-05-23 17:43:17.764 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:17.764 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 755 (0x7f431bfd6990) [ CSSD]2013-05-23 17:43:17.764 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6990 (RPC#755) tag(2f3002d) sent to node 1 [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02f3002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015421, gin 1, member 0 [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2321ea0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6990 (RPC#755) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6990, ret 0, client 0x2321d30 member 0x22e50a0 [ CSSD]2013-05-23 17:43:17.764 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321d30 [ CSSD]2013-05-23 17:43:17.765 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 755 [ CSSD]2013-05-23 17:43:17.770 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321d30) [ CSSD]2013-05-23 17:43:17.770 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:17.770 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e50a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:17.770 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 756 (0x7f431bfd6a10) [ CSSD]2013-05-23 17:43:17.770 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6a10 (RPC#756) tag(2f4002d) sent to node 1 [ CSSD]2013-05-23 17:43:17.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02f4002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:17.771 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:17.771 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6a10 (RPC#756) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:17.771 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6a10, ret 0, client 0x2321d30 member 0x22e50a0 [ CSSD]2013-05-23 17:43:17.771 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 756 [ CSSD]2013-05-23 17:43:17.771 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e50a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:43:17.973 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e51c0 [ CSSD]2013-05-23 17:43:17.973 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e51c0) con(0x22e3890) skgpid ospid 6022 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:43:18.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:18.246 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:19.052 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:20.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:20.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:21.058 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:21.676 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:21.676 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:22.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:22.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:23.062 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:24.269 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:24.270 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:25.074 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:26.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:26.280 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:26.515 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:43:26.515 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 757 (0x7f431bfd6a90) [ CSSD]2013-05-23 17:43:26.515 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6a90 (RPC#757) tag(2f5002d) sent to node 1 [ CSSD]2013-05-23 17:43:26.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:26.516 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 757 (0x7f431bfd6a90) [ CSSD]2013-05-23 17:43:26.516 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6a90 (RPC#757) state 6, flags 0x202 [ CSSD]2013-05-23 17:43:26.516 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6a90, ret 0, client (nil) [ CSSD]2013-05-23 17:43:26.516 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 757 [ CSSD]2013-05-23 17:43:26.678 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:26.678 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:27.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:27.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:27.718 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015422, gin 1, member 0 [ CSSD]2013-05-23 17:43:27.718 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:43:27.718 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:43:27.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:27.725 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:43:27.725 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:43:28.305 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:28.305 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:29.152 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:30.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:30.308 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:30.681 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:30.681 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:43:31.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:32.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:32.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:33.005 [1191196992] >TRACE: clssgmAllocProc: (0x22e50f0) allocated [ CSSD]2013-05-23 17:43:33.005 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:43:33.006 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e50f0 - 1,2,3 [ CSSD]2013-05-23 17:43:33.006 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e37c0) proc(0x22e50f0) pid(6170) version 11:1:1:4 [ CSSD]2013-05-23 17:43:33.022 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e50f0), client(1/0x7f431c13f730) [ CSSD]2013-05-23 17:43:33.022 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:43:33.022 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f730 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:43:33.022 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:33.022 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 758 (0x7f431bfd6b10) [ CSSD]2013-05-23 17:43:33.022 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6b10 (RPC#758) tag(2f6002d) sent to node 1 [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02f6002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015423, gin 1, member 0 [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6b10 (RPC#758) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6b10, ret 0, client 0x7f431c13f730 member 0x7f431c1248c0 [ CSSD]2013-05-23 17:43:33.022 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f730 [ CSSD]2013-05-23 17:43:33.023 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 758 [ CSSD]2013-05-23 17:43:33.029 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f730) [ CSSD]2013-05-23 17:43:33.029 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f730), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:33.029 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1248c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:33.029 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 759 (0x7f431bfd6b90) [ CSSD]2013-05-23 17:43:33.029 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6b90 (RPC#759) tag(2f7002d) sent to node 1 [ CSSD]2013-05-23 17:43:33.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02f7002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:33.029 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:33.029 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6b90 (RPC#759) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:33.029 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6b90, ret 0, client 0x7f431c13f730 member 0x7f431c1248c0 [ CSSD]2013-05-23 17:43:33.029 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 759 [ CSSD]2013-05-23 17:43:33.029 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1248c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:43:33.163 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:33.232 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e50f0 [ CSSD]2013-05-23 17:43:33.232 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e50f0) con(0x22e37c0) skgpid ospid 6170 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:43:34.338 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:34.338 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:34.375 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x2318e90), client(2/0x7f431c13f730) [ CSSD]2013-05-23 17:43:34.375 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c13f730) [ CSSD]2013-05-23 17:43:34.375 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:43:34.375 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:43:35.176 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:35.683 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:35.683 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:36.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:36.371 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:36.540 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:43:36.540 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 760 (0x7f431bfd6c10) [ CSSD]2013-05-23 17:43:36.541 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6c10 (RPC#760) tag(2f8002d) sent to node 1 [ CSSD]2013-05-23 17:43:36.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:36.541 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 760 (0x7f431bfd6c10) [ CSSD]2013-05-23 17:43:36.541 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6c10 (RPC#760) state 6, flags 0x202 [ CSSD]2013-05-23 17:43:36.541 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6c10, ret 0, client (nil) [ CSSD]2013-05-23 17:43:36.541 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 760 [ CSSD]2013-05-23 17:43:36.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:36.938 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015424, gin 1, member 0 [ CSSD]2013-05-23 17:43:36.938 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:43:36.938 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:43:36.945 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:36.945 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:43:36.945 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:43:37.200 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:38.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:38.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:39.203 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:40.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:40.388 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:40.686 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:40.686 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:40.846 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c095a70) allocated [ CSSD]2013-05-23 17:43:40.847 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:43:40.847 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c095a70 - 1,2,3 [ CSSD]2013-05-23 17:43:40.847 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c095a70) pid(6269) version 11:1:1:4 [ CSSD]2013-05-23 17:43:40.852 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c095a70), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:43:40.852 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:43:40.853 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 17:43:40.853 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:40.853 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 761 (0x7f431bfd6c90) [ CSSD]2013-05-23 17:43:40.853 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6c90 (RPC#761) tag(2f9002d) sent to node 1 [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02f9002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015425, gin 1, member 0 [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6c90 (RPC#761) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6c90, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:43:40.853 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 761 [ CSSD]2013-05-23 17:43:40.859 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:43:40.859 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:40.859 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:40.859 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 762 (0x7f431bfd6d10) [ CSSD]2013-05-23 17:43:40.859 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6d10 (RPC#762) tag(2fa002d) sent to node 1 [ CSSD]2013-05-23 17:43:40.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02fa002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:40.860 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:40.860 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6d10 (RPC#762) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:40.860 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6d10, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:43:40.860 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 762 [ CSSD]2013-05-23 17:43:40.860 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:43:41.062 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c095a70 [ CSSD]2013-05-23 17:43:41.062 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c095a70) con(0x7f431c12a550) skgpid ospid 6269 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:43:41.208 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:42.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:42.393 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:42.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:42.968 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015426, gin 1, member 0 [ CSSD]2013-05-23 17:43:42.968 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:43:42.968 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:43:42.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:42.975 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:43:42.975 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:43:43.260 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:43.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:43.467 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015427, gin 1, member 0 [ CSSD]2013-05-23 17:43:43.467 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c1215d0/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:43:43.467 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:43:43.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:43.474 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:43:43.474 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:43:43.598 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:43:43.599 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:43:43.600 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:43:43.600 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:43:43.602 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:43:43.602 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:43:44.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:44.418 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:45.286 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:45.688 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:45.688 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:46.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:46.472 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:46.567 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:43:46.567 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 763 (0x7f431bfd6d90) [ CSSD]2013-05-23 17:43:46.567 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6d90 (RPC#763) tag(2fb002d) sent to node 1 [ CSSD]2013-05-23 17:43:46.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:46.567 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 763 (0x7f431bfd6d90) [ CSSD]2013-05-23 17:43:46.567 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6d90 (RPC#763) state 6, flags 0x202 [ CSSD]2013-05-23 17:43:46.567 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6d90, ret 0, client (nil) [ CSSD]2013-05-23 17:43:46.568 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 763 [ CSSD]2013-05-23 17:43:47.309 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:48.266 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135290) allocated [ CSSD]2013-05-23 17:43:48.266 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:43:48.267 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135290 - 1,2,3 [ CSSD]2013-05-23 17:43:48.267 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c135290) pid(6354) version 11:1:1:4 [ CSSD]2013-05-23 17:43:48.272 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c135290), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:43:48.272 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:43:48.272 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:43:48.272 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:48.272 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 764 (0x7f431bfd6e10) [ CSSD]2013-05-23 17:43:48.272 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6e10 (RPC#764) tag(2fc002d) sent to node 1 [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02fc002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015428, gin 1, member 0 [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1215d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6e10 (RPC#764) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6e10, ret 0, client 0x7f431c12e6d0 member 0x7f431c134d30 [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:43:48.273 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 764 [ CSSD]2013-05-23 17:43:48.278 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:43:48.278 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:48.278 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134d30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:43:48.278 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 765 (0x7f431bfd6e90) [ CSSD]2013-05-23 17:43:48.279 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6e90 (RPC#765) tag(2fd002d) sent to node 1 [ CSSD]2013-05-23 17:43:48.279 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(02fd002d) incarnation(174631051) [ CSSD]2013-05-23 17:43:48.279 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:43:48.279 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6e90 (RPC#765) state 6, flags 0x100 [ CSSD]2013-05-23 17:43:48.279 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd6e90, ret 0, client 0x7f431c12e6d0 member 0x7f431c134d30 [ CSSD]2013-05-23 17:43:48.279 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 765 [ CSSD]2013-05-23 17:43:48.279 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134d30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:43:48.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:48.478 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:48.481 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135290 [ CSSD]2013-05-23 17:43:48.481 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135290) con(0x7f431c13f020) skgpid ospid 6354 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:43:49.364 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:50.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:50.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:50.691 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:50.691 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:43:51.369 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:52.495 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:52.495 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:53.380 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:54.501 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:54.501 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:54.693 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:54.694 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:43:55.160 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c092940) [ CSSD]2013-05-23 17:43:55.160 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c092940), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:43:55.160 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c092940), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:43:55.160 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:43:55.160 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:43:55.160 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e6e20) skgpid 482 ospid 482 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:43:55.386 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:56.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:56.511 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:56.590 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:43:56.590 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 766 (0x7f431bfd6f10) [ CSSD]2013-05-23 17:43:56.591 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6f10 (RPC#766) tag(2fe002d) sent to node 1 [ CSSD]2013-05-23 17:43:56.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:56.591 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 766 (0x7f431bfd6f10) [ CSSD]2013-05-23 17:43:56.591 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6f10 (RPC#766) state 6, flags 0x202 [ CSSD]2013-05-23 17:43:56.591 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd6f10, ret 0, client (nil) [ CSSD]2013-05-23 17:43:56.591 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 766 [ CSSD]2013-05-23 17:43:57.411 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:58.237 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:58.237 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015429, gin 1, member 0 [ CSSD]2013-05-23 17:43:58.237 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c134ba0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:43:58.237 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:43:58.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:58.244 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:43:58.244 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:43:58.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:43:58.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:43:59.484 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:43:59.696 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:43:59.696 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:00.570 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:00.570 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:01.490 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:02.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:02.595 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:03.502 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:03.514 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:44:03.514 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:44:03.515 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:44:03.515 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1355f0) pid(6505) version 11:1:1:4 [ CSSD]2013-05-23 17:44:03.521 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c1355f0), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:03.521 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:03.521 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 17:44:03.521 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:03.521 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 767 (0x7f431bfd6f90) [ CSSD]2013-05-23 17:44:03.522 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd6f90 (RPC#767) tag(2ff002d) sent to node 1 [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(02ff002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015430, gin 1, member 0 [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134ba0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd6f90 (RPC#767) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd6f90, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e6d60 [ CSSD]2013-05-23 17:44:03.522 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:44:03.523 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 767 [ CSSD]2013-05-23 17:44:03.528 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:03.528 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:03.528 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:03.529 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 768 (0x7f431bfd7010) [ CSSD]2013-05-23 17:44:03.529 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7010 (RPC#768) tag(300002d) sent to node 1 [ CSSD]2013-05-23 17:44:03.529 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0300002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:03.529 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:03.529 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7010 (RPC#768) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:03.529 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7010, ret 0, client 0x7f431c12e6d0 member 0x7f431c0e6d60 [ CSSD]2013-05-23 17:44:03.529 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 768 [ CSSD]2013-05-23 17:44:03.529 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:44:03.731 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:44:03.731 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c0e6720) skgpid ospid 6505 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:44:04.601 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:04.601 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:04.699 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:04.699 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:05.508 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:06.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:06.613 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:06.617 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:44:06.617 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 769 (0x7f431bfd7090) [ CSSD]2013-05-23 17:44:06.618 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7090 (RPC#769) tag(301002d) sent to node 1 [ CSSD]2013-05-23 17:44:06.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:06.618 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 769 (0x7f431bfd7090) [ CSSD]2013-05-23 17:44:06.618 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7090 (RPC#769) state 6, flags 0x202 [ CSSD]2013-05-23 17:44:06.618 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7090, ret 0, client (nil) [ CSSD]2013-05-23 17:44:06.618 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 769 [ CSSD]2013-05-23 17:44:07.533 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:08.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:08.618 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:09.564 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:09.701 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:09.702 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:10.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:10.670 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:11.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:12.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:12.676 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:13.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:13.486 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015431, gin 1, member 0 [ CSSD]2013-05-23 17:44:13.486 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c134ba0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:44:13.486 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:44:13.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:13.492 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:44:13.492 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1248c0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:44:13.582 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:14.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:14.700 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:14.705 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:14.705 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:15.589 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:16.640 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:44:16.640 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 770 (0x7f431bfd7110) [ CSSD]2013-05-23 17:44:16.640 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7110 (RPC#770) tag(302002d) sent to node 1 [ CSSD]2013-05-23 17:44:16.641 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:16.641 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 770 (0x7f431bfd7110) [ CSSD]2013-05-23 17:44:16.641 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7110 (RPC#770) state 6, flags 0x202 [ CSSD]2013-05-23 17:44:16.641 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7110, ret 0, client (nil) [ CSSD]2013-05-23 17:44:16.641 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 770 [ CSSD]2013-05-23 17:44:16.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:16.712 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:17.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:18.707 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:18.707 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:44:18.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:18.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:18.763 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:44:18.763 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:44:18.764 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:44:18.764 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1355f0) pid(6667) version 11:1:1:4 [ CSSD]2013-05-23 17:44:18.770 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c1355f0), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:18.770 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:18.770 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:44:18.770 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:18.770 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 771 (0x7f431bfd7190) [ CSSD]2013-05-23 17:44:18.770 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7190 (RPC#771) tag(303002d) sent to node 1 [ CSSD]2013-05-23 17:44:18.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0303002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:18.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015432, gin 1, member 0 [ CSSD]2013-05-23 17:44:18.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c134ba0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:44:18.771 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:44:18.771 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:44:18.771 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7190 (RPC#771) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:18.771 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7190, ret 0, client 0x7f431c12e6d0 member 0x7f431c10a490 [ CSSD]2013-05-23 17:44:18.771 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:44:18.771 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 771 [ CSSD]2013-05-23 17:44:18.777 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:18.777 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:18.777 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:18.777 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 772 (0x7f431bfd7210) [ CSSD]2013-05-23 17:44:18.777 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7210 (RPC#772) tag(304002d) sent to node 1 [ CSSD]2013-05-23 17:44:18.777 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0304002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:18.777 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:18.777 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7210 (RPC#772) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:18.777 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7210, ret 0, client 0x7f431c12e6d0 member 0x7f431c10a490 [ CSSD]2013-05-23 17:44:18.778 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 772 [ CSSD]2013-05-23 17:44:18.778 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:44:18.980 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:44:18.980 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12f0c0) skgpid ospid 6667 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:44:19.605 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:20.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:20.771 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:21.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:22.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:22.777 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:23.662 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:23.710 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:23.710 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:24.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:24.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:25.668 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:26.664 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:44:26.664 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 773 (0x7f431bfd7290) [ CSSD]2013-05-23 17:44:26.664 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7290 (RPC#773) tag(305002d) sent to node 1 [ CSSD]2013-05-23 17:44:26.665 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:26.665 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 773 (0x7f431bfd7290) [ CSSD]2013-05-23 17:44:26.665 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7290 (RPC#773) state 6, flags 0x202 [ CSSD]2013-05-23 17:44:26.665 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7290, ret 0, client (nil) [ CSSD]2013-05-23 17:44:26.665 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 773 [ CSSD]2013-05-23 17:44:26.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:26.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:27.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:28.306 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:44:28.306 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:44:28.307 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:44:28.307 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1355f0) pid(6446/6446) version 11:1:1:4 [ CSSD]2013-05-23 17:44:28.322 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c1355f0), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:28.322 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:44:28.322 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:44:28.322 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:44:28.712 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:28.712 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:28.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:28.746 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015433, gin 1, member 0 [ CSSD]2013-05-23 17:44:28.746 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c134ba0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:44:28.746 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:44:28.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:28.752 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:44:28.752 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e5840) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:44:28.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:28.820 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:29.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:29.944 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c1355f0), client(2/0x7f431c135350) [ CSSD]2013-05-23 17:44:29.944 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c135350) [ CSSD]2013-05-23 17:44:29.944 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:44:29.945 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 17:44:30.767 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:44:30.768 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:44:30.769 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:44:30.769 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:44:30.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:30.851 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015434, gin 1, member 0 [ CSSD]2013-05-23 17:44:30.851 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c134ba0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:44:30.851 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:44:30.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:30.858 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:44:30.858 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c11fd30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:44:30.870 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:30.870 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:31.711 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:32.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:32.876 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:33.715 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:33.715 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:33.763 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:33.809 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13ea10) allocated [ CSSD]2013-05-23 17:44:33.810 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:44:33.811 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13ea10 - 1,2,3 [ CSSD]2013-05-23 17:44:33.811 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c13ea10) pid(6444/6444) version 11:1:1:4 [ CSSD]2013-05-23 17:44:33.820 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c132e60) allocated [ CSSD]2013-05-23 17:44:33.821 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:44:33.821 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c132e60 - 1,2,3 [ CSSD]2013-05-23 17:44:33.821 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c132e60) pid(6442/6442) version 11:1:1:4 [ CSSD]2013-05-23 17:44:33.837 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13ea10), client(1/0x7f431c108e70) [ CSSD]2013-05-23 17:44:33.837 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:44:33.837 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:44:33.837 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 17:44:33.838 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c132e60), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 17:44:33.838 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 17:44:33.838 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:44:33.838 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 17:44:33.847 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134730) allocated [ CSSD]2013-05-23 17:44:33.847 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:44:33.848 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134730 - 1,2,3 [ CSSD]2013-05-23 17:44:33.848 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c121820) proc(0x7f431c134730) pid(5743/5743) version 11:1:1:4 [ CSSD]2013-05-23 17:44:33.863 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c134730), client(1/0x7f431c134690) [ CSSD]2013-05-23 17:44:33.863 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c134690) [ CSSD]2013-05-23 17:44:33.863 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:44:33.863 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 35 [ CSSD]2013-05-23 17:44:34.019 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092680) allocated [ CSSD]2013-05-23 17:44:34.021 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:44:34.021 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092680 - 1,2,3 [ CSSD]2013-05-23 17:44:34.021 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa20) proc(0x7f431c092680) pid(6841) version 11:1:1:4 [ CSSD]2013-05-23 17:44:34.027 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c092680), client(1/0x7f431c0925e0) [ CSSD]2013-05-23 17:44:34.027 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0925e0) [ CSSD]2013-05-23 17:44:34.027 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0925e0 with con 0x7f431c0928e0, requested num -1 [ CSSD]2013-05-23 17:44:34.027 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:34.027 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 774 (0x7f431bfd7310) [ CSSD]2013-05-23 17:44:34.027 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7310 (RPC#774) tag(306002d) sent to node 1 [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0306002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015435, gin 1, member 0 [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c136700/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7310 (RPC#774) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7310, ret 0, client 0x7f431c0925e0 member 0x7f431c136a00 [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0925e0 [ CSSD]2013-05-23 17:44:34.028 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 774 [ CSSD]2013-05-23 17:44:34.034 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0925e0) [ CSSD]2013-05-23 17:44:34.034 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0925e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:34.034 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:34.034 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 775 (0x7f431bfd7390) [ CSSD]2013-05-23 17:44:34.034 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7390 (RPC#775) tag(307002d) sent to node 1 [ CSSD]2013-05-23 17:44:34.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0307002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:34.034 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:34.034 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7390 (RPC#775) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:34.034 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7390, ret 0, client 0x7f431c0925e0 member 0x7f431c136a00 [ CSSD]2013-05-23 17:44:34.035 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 775 [ CSSD]2013-05-23 17:44:34.035 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:44:34.237 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092680 [ CSSD]2013-05-23 17:44:34.237 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092680) con(0x7f431c13fa20) skgpid ospid 6841 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:44:34.888 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:34.888 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:35.520 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c134730), client(2/0x7f431c11ab50) [ CSSD]2013-05-23 17:44:35.520 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c11ab50) [ CSSD]2013-05-23 17:44:35.520 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:44:35.520 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 28 [ CSSD]2013-05-23 17:44:35.769 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:36.442 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c132e60), client(2/0x7f431c1306d0) [ CSSD]2013-05-23 17:44:36.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c1306d0) [ CSSD]2013-05-23 17:44:36.442 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:44:36.442 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 29 [ CSSD]2013-05-23 17:44:36.628 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c13ea10), client(2/0x7f431c130630) [ CSSD]2013-05-23 17:44:36.628 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c130630) [ CSSD]2013-05-23 17:44:36.628 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:44:36.628 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 30 [ CSSD]2013-05-23 17:44:36.693 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:44:36.693 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 776 (0x7f431bfd7410) [ CSSD]2013-05-23 17:44:36.693 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7410 (RPC#776) tag(308002d) sent to node 1 [ CSSD]2013-05-23 17:44:36.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:36.694 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 776 (0x7f431bfd7410) [ CSSD]2013-05-23 17:44:36.694 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7410 (RPC#776) state 6, flags 0x202 [ CSSD]2013-05-23 17:44:36.694 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7410, ret 0, client (nil) [ CSSD]2013-05-23 17:44:36.694 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 776 [ CSSD]2013-05-23 17:44:36.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:36.895 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:37.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:37.200 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015436, gin 1, member 0 [ CSSD]2013-05-23 17:44:37.200 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c090cf0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:44:37.200 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:44:37.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:37.207 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:44:37.207 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c1302e0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:44:37.775 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:38.718 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:38.718 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:38.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:38.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:39.866 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:40.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:40.920 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:41.099 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136d30) allocated [ CSSD]2013-05-23 17:44:41.100 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:44:41.101 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136d30 - 1,2,3 [ CSSD]2013-05-23 17:44:41.101 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0910d0) proc(0x7f431c136d30) pid(6934) version 11:1:1:4 [ CSSD]2013-05-23 17:44:41.106 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c136d30), client(1/0x7f431c136c90) [ CSSD]2013-05-23 17:44:41.106 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:44:41.106 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c136c90 with con 0x7f431c0927f0, requested num -1 [ CSSD]2013-05-23 17:44:41.106 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:41.106 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 777 (0x7f431bfd7490) [ CSSD]2013-05-23 17:44:41.106 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7490 (RPC#777) tag(309002d) sent to node 1 [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0309002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015437, gin 1, member 0 [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137470/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7490 (RPC#777) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7490, ret 0, client 0x7f431c136c90 member 0x7f431c137770 [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c136c90 [ CSSD]2013-05-23 17:44:41.107 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 777 [ CSSD]2013-05-23 17:44:41.112 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:44:41.112 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c136c90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:41.112 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137770) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:41.112 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 778 (0x7f431bfd7510) [ CSSD]2013-05-23 17:44:41.112 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7510 (RPC#778) tag(30a002d) sent to node 1 [ CSSD]2013-05-23 17:44:41.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(030a002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:41.113 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:41.113 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7510 (RPC#778) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:41.113 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7510, ret 0, client 0x7f431c136c90 member 0x7f431c137770 [ CSSD]2013-05-23 17:44:41.113 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 778 [ CSSD]2013-05-23 17:44:41.113 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137770) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:44:41.316 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136d30 [ CSSD]2013-05-23 17:44:41.316 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136d30) con(0x7f431c0910d0) skgpid ospid 6934 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:44:41.872 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:42.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:42.926 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:43.721 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:43.721 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:43.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:44.000 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:44.000 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015438, gin 1, member 0 [ CSSD]2013-05-23 17:44:44.001 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:44:44.001 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:44:44.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:44.006 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:44:44.006 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:44:44.929 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:44.929 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:45.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:46.716 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:44:46.716 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 779 (0x7f431bfd7590) [ CSSD]2013-05-23 17:44:46.717 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7590 (RPC#779) tag(30b002d) sent to node 1 [ CSSD]2013-05-23 17:44:46.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:46.717 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 779 (0x7f431bfd7590) [ CSSD]2013-05-23 17:44:46.717 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7590 (RPC#779) state 6, flags 0x202 [ CSSD]2013-05-23 17:44:46.717 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7590, ret 0, client (nil) [ CSSD]2013-05-23 17:44:46.717 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 779 [ CSSD]2013-05-23 17:44:46.955 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:46.955 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:47.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:48.724 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:48.724 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:48.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:48.962 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:49.277 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136d30) allocated [ CSSD]2013-05-23 17:44:49.278 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:44:49.279 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136d30 - 1,2,3 [ CSSD]2013-05-23 17:44:49.279 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0910d0) proc(0x7f431c136d30) pid(7009) version 11:1:1:4 [ CSSD]2013-05-23 17:44:49.284 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c136d30), client(1/0x7f431c136c90) [ CSSD]2013-05-23 17:44:49.284 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:44:49.284 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c136c90 with con 0x7f431c0927f0, requested num -1 [ CSSD]2013-05-23 17:44:49.284 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:49.284 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 780 (0x7f431bfd7610) [ CSSD]2013-05-23 17:44:49.284 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7610 (RPC#780) tag(30c002d) sent to node 1 [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(030c002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015439, gin 1, member 0 [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1374e0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7610 (RPC#780) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7610, ret 0, client 0x7f431c136c90 member 0x7f431c1377e0 [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c136c90 [ CSSD]2013-05-23 17:44:49.285 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 780 [ CSSD]2013-05-23 17:44:49.291 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:44:49.291 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c136c90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:49.291 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1377e0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:44:49.291 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 781 (0x7f431bfd7690) [ CSSD]2013-05-23 17:44:49.291 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7690 (RPC#781) tag(30d002d) sent to node 1 [ CSSD]2013-05-23 17:44:49.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(030d002d) incarnation(174631051) [ CSSD]2013-05-23 17:44:49.291 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:44:49.291 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7690 (RPC#781) state 6, flags 0x100 [ CSSD]2013-05-23 17:44:49.291 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7690, ret 0, client 0x7f431c136c90 member 0x7f431c1377e0 [ CSSD]2013-05-23 17:44:49.291 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 781 [ CSSD]2013-05-23 17:44:49.291 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1377e0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:44:49.494 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136d30 [ CSSD]2013-05-23 17:44:49.494 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136d30) con(0x7f431c0910d0) skgpid ospid 7009 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:44:49.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:50.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:50.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:51.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:53.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:53.016 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:53.727 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:53.727 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:53.964 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:55.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:55.110 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:55.990 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:56.740 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:44:56.741 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 782 (0x7f431bfd7710) [ CSSD]2013-05-23 17:44:56.741 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7710 (RPC#782) tag(30e002d) sent to node 1 [ CSSD]2013-05-23 17:44:56.741 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:56.741 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 782 (0x7f431bfd7710) [ CSSD]2013-05-23 17:44:56.741 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7710 (RPC#782) state 6, flags 0x202 [ CSSD]2013-05-23 17:44:56.741 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7710, ret 0, client (nil) [ CSSD]2013-05-23 17:44:56.741 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 782 [ CSSD]2013-05-23 17:44:57.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:57.116 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:58.015 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:44:58.729 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:44:58.729 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:44:59.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:59.161 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:44:59.249 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:59.249 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015440, gin 1, member 0 [ CSSD]2013-05-23 17:44:59.250 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:44:59.250 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:44:59.256 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:44:59.257 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:44:59.257 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:45:00.018 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:01.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:01.193 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:02.023 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:03.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:03.199 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:03.731 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:03.732 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:04.094 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:04.525 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136d30) allocated [ CSSD]2013-05-23 17:45:04.525 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:45:04.526 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136d30 - 1,2,3 [ CSSD]2013-05-23 17:45:04.526 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091190) proc(0x7f431c136d30) pid(7146) version 11:1:1:4 [ CSSD]2013-05-23 17:45:04.531 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c136d30), client(1/0x7f431c136c90) [ CSSD]2013-05-23 17:45:04.531 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:45:04.531 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c136c90 with con 0x7f431c136fa0, requested num -1 [ CSSD]2013-05-23 17:45:04.531 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:04.531 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 783 (0x7f431bfd7790) [ CSSD]2013-05-23 17:45:04.531 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7790 (RPC#783) tag(30f002d) sent to node 1 [ CSSD]2013-05-23 17:45:04.531 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(030f002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:04.531 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015441, gin 1, member 0 [ CSSD]2013-05-23 17:45:04.531 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137590/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:45:04.531 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:45:04.532 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:45:04.532 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7790 (RPC#783) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:04.532 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7790, ret 0, client 0x7f431c136c90 member 0x7f431c137890 [ CSSD]2013-05-23 17:45:04.532 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c136c90 [ CSSD]2013-05-23 17:45:04.532 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 783 [ CSSD]2013-05-23 17:45:04.537 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:45:04.537 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c136c90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:04.537 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137890) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:04.537 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 784 (0x7f431bfd7810) [ CSSD]2013-05-23 17:45:04.537 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7810 (RPC#784) tag(310002d) sent to node 1 [ CSSD]2013-05-23 17:45:04.537 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0310002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:04.537 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:04.537 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7810 (RPC#784) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:04.537 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7810, ret 0, client 0x7f431c136c90 member 0x7f431c137890 [ CSSD]2013-05-23 17:45:04.538 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 784 [ CSSD]2013-05-23 17:45:04.538 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137890) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:45:04.740 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136d30 [ CSSD]2013-05-23 17:45:04.740 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136d30) con(0x7f431c091190) skgpid ospid 7146 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:05.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:05.209 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:06.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:06.765 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:45:06.765 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 785 (0x7f431bfd7890) [ CSSD]2013-05-23 17:45:06.766 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7890 (RPC#785) tag(311002d) sent to node 1 [ CSSD]2013-05-23 17:45:06.766 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:06.766 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 785 (0x7f431bfd7890) [ CSSD]2013-05-23 17:45:06.766 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7890 (RPC#785) state 6, flags 0x202 [ CSSD]2013-05-23 17:45:06.766 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7890, ret 0, client (nil) [ CSSD]2013-05-23 17:45:06.766 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 785 [ CSSD]2013-05-23 17:45:07.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:07.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:08.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:08.421 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136d30) allocated [ CSSD]2013-05-23 17:45:08.422 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:45:08.423 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136d30 - 1,2,3 [ CSSD]2013-05-23 17:45:08.423 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091190) proc(0x7f431c136d30) pid(7235) version 11:1:1:4 [ CSSD]2013-05-23 17:45:08.734 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:08.734 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:08.996 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136d30 [ CSSD]2013-05-23 17:45:08.996 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136d30) con(0x7f431c091190) skgpid ospid 7235 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:09.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:09.221 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:10.118 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:11.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:11.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:12.124 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:12.736 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:12.736 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:45:13.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:13.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:14.175 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:14.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:14.511 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015442, gin 1, member 0 [ CSSD]2013-05-23 17:45:14.511 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e3300/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:45:14.511 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:45:14.518 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:14.518 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:45:14.518 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:45:15.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:15.291 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:16.180 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:16.792 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:45:16.792 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 786 (0x7f431bfd7910) [ CSSD]2013-05-23 17:45:16.792 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7910 (RPC#786) tag(312002d) sent to node 1 [ CSSD]2013-05-23 17:45:16.793 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:16.793 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 786 (0x7f431bfd7910) [ CSSD]2013-05-23 17:45:16.793 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7910 (RPC#786) state 6, flags 0x202 [ CSSD]2013-05-23 17:45:16.793 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7910, ret 0, client (nil) [ CSSD]2013-05-23 17:45:16.793 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 786 [ CSSD]2013-05-23 17:45:17.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:17.297 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:17.739 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:17.739 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:18.205 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:19.322 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:19.322 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:19.771 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136d30) allocated [ CSSD]2013-05-23 17:45:19.772 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:45:19.773 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136d30 - 1,2,3 [ CSSD]2013-05-23 17:45:19.773 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091190) proc(0x7f431c136d30) pid(7407) version 11:1:1:4 [ CSSD]2013-05-23 17:45:19.778 [1191196992] >TRACE: clssgmRegisterClient: proc(64/0x7f431c136d30), client(1/0x7f431c136c90) [ CSSD]2013-05-23 17:45:19.778 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:45:19.778 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c136c90 with con 0x7f431c136fa0, requested num -1 [ CSSD]2013-05-23 17:45:19.778 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:19.778 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 787 (0x7f431bfd7990) [ CSSD]2013-05-23 17:45:19.778 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7990 (RPC#787) tag(313002d) sent to node 1 [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0313002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015443, gin 1, member 0 [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137590/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7990 (RPC#787) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7990, ret 0, client 0x7f431c136c90 member 0x7f431c137890 [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c136c90 [ CSSD]2013-05-23 17:45:19.779 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 787 [ CSSD]2013-05-23 17:45:19.785 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c136c90) [ CSSD]2013-05-23 17:45:19.785 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c136c90), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:19.785 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137890) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:19.785 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 788 (0x7f431bfd7a10) [ CSSD]2013-05-23 17:45:19.785 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7a10 (RPC#788) tag(314002d) sent to node 1 [ CSSD]2013-05-23 17:45:19.786 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0314002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:19.786 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:19.786 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7a10 (RPC#788) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:19.786 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7a10, ret 0, client 0x7f431c136c90 member 0x7f431c137890 [ CSSD]2013-05-23 17:45:19.786 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 788 [ CSSD]2013-05-23 17:45:19.786 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137890) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:45:19.988 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136d30 [ CSSD]2013-05-23 17:45:19.988 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136d30) con(0x7f431c091190) skgpid ospid 7407 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:20.215 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:20.537 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:45:20.537 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:45:20.537 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:45:20.537 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 34 [ CSSD]2013-05-23 17:45:20.537 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed60 [ CSSD]2013-05-23 17:45:20.537 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed60) con(0x22dffa0) skgpid 1702 ospid 1702 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:20.552 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2302d30) [ CSSD]2013-05-23 17:45:20.552 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2302d30), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:45:20.552 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2302d30), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:45:20.552 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 29 [ CSSD]2013-05-23 17:45:20.562 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:45:20.562 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:45:20.562 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:45:20.562 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 17:45:20.562 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:45:20.562 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0cf710) skgpid 1704 ospid 1704 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:21.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:21.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:22.221 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:22.742 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:22.742 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:23.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:23.340 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:24.276 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:25.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:25.391 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:25.425 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c082370) allocated [ CSSD]2013-05-23 17:45:25.425 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:45:25.426 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c082370 - 1,2,3 [ CSSD]2013-05-23 17:45:25.426 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0cf710) proc(0x7f431c082370) pid(7460/7460) version 11:1:1:4 [ CSSD]2013-05-23 17:45:25.431 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c082370), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:45:25.431 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:45:25.431 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:45:25.431 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 17:45:26.282 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:26.815 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:45:26.816 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 789 (0x7f431bfd7a90) [ CSSD]2013-05-23 17:45:26.816 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7a90 (RPC#789) tag(315002d) sent to node 1 [ CSSD]2013-05-23 17:45:26.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:26.816 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 789 (0x7f431bfd7a90) [ CSSD]2013-05-23 17:45:26.816 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7a90 (RPC#789) state 6, flags 0x202 [ CSSD]2013-05-23 17:45:26.816 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7a90, ret 0, client (nil) [ CSSD]2013-05-23 17:45:26.816 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 789 [ CSSD]2013-05-23 17:45:27.071 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c082370), client(2/0x7f431c130b40) [ CSSD]2013-05-23 17:45:27.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c130b40) [ CSSD]2013-05-23 17:45:27.072 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:45:27.072 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 30 [ CSSD]2013-05-23 17:45:27.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:27.397 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:27.745 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:27.745 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:28.293 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:29.403 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:29.403 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:29.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:29.762 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015444, gin 1, member 0 [ CSSD]2013-05-23 17:45:29.762 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:45:29.762 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:45:29.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:29.768 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:45:29.768 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:45:30.299 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:31.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:31.413 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:32.305 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:32.748 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:32.748 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:33.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:33.418 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:34.316 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:35.020 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092560) allocated [ CSSD]2013-05-23 17:45:35.020 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:45:35.021 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092560 - 1,2,3 [ CSSD]2013-05-23 17:45:35.021 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091660) proc(0x7f431c092560) pid(7570) version 11:1:1:4 [ CSSD]2013-05-23 17:45:35.026 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c092560), client(1/0x7f431c137890) [ CSSD]2013-05-23 17:45:35.026 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137890) [ CSSD]2013-05-23 17:45:35.026 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137890 with con 0x7f431c092460, requested num -1 [ CSSD]2013-05-23 17:45:35.026 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:35.026 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 790 (0x7f431bfd7b10) [ CSSD]2013-05-23 17:45:35.026 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7b10 (RPC#790) tag(316002d) sent to node 1 [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0316002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015445, gin 1, member 0 [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137a00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7b10 (RPC#790) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7b10, ret 0, client 0x7f431c137890 member 0x7f431c137d00 [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137890 [ CSSD]2013-05-23 17:45:35.027 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 790 [ CSSD]2013-05-23 17:45:35.034 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137890) [ CSSD]2013-05-23 17:45:35.034 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137890), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:35.034 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137d00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:35.034 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 791 (0x7f431bfd7b90) [ CSSD]2013-05-23 17:45:35.034 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7b90 (RPC#791) tag(317002d) sent to node 1 [ CSSD]2013-05-23 17:45:35.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0317002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:35.034 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:35.034 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7b90 (RPC#791) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:35.034 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7b90, ret 0, client 0x7f431c137890 member 0x7f431c137d00 [ CSSD]2013-05-23 17:45:35.035 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 791 [ CSSD]2013-05-23 17:45:35.035 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137d00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:45:35.237 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092560 [ CSSD]2013-05-23 17:45:35.237 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092560) con(0x7f431c091660) skgpid ospid 7570 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:35.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:35.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:36.321 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:36.842 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:45:36.842 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 792 (0x7f431bfd7c10) [ CSSD]2013-05-23 17:45:36.842 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7c10 (RPC#792) tag(318002d) sent to node 1 [ CSSD]2013-05-23 17:45:36.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:36.842 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 792 (0x7f431bfd7c10) [ CSSD]2013-05-23 17:45:36.842 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7c10 (RPC#792) state 6, flags 0x202 [ CSSD]2013-05-23 17:45:36.842 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7c10, ret 0, client (nil) [ CSSD]2013-05-23 17:45:36.843 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 792 [ CSSD]2013-05-23 17:45:37.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:37.461 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015446, gin 1, member 0 [ CSSD]2013-05-23 17:45:37.461 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:45:37.461 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:45:37.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:37.467 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:45:37.467 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:45:37.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:37.516 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:37.751 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:37.751 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:38.374 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:39.522 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:39.522 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:40.418 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:41.348 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092560) allocated [ CSSD]2013-05-23 17:45:41.349 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:45:41.350 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092560 - 1,2,3 [ CSSD]2013-05-23 17:45:41.350 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091660) proc(0x7f431c092560) pid(7664) version 11:1:1:4 [ CSSD]2013-05-23 17:45:41.355 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c092560), client(1/0x7f431c137890) [ CSSD]2013-05-23 17:45:41.355 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137890) [ CSSD]2013-05-23 17:45:41.355 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137890 with con 0x7f431c092460, requested num -1 [ CSSD]2013-05-23 17:45:41.355 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:41.355 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 793 (0x7f431bfd7c90) [ CSSD]2013-05-23 17:45:41.356 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7c90 (RPC#793) tag(319002d) sent to node 1 [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0319002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015447, gin 1, member 0 [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137a00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7c90 (RPC#793) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7c90, ret 0, client 0x7f431c137890 member 0x7f431c137d00 [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137890 [ CSSD]2013-05-23 17:45:41.356 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 793 [ CSSD]2013-05-23 17:45:41.362 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137890) [ CSSD]2013-05-23 17:45:41.362 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137890), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:41.362 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137d00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:41.362 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 794 (0x7f431bfd7d10) [ CSSD]2013-05-23 17:45:41.362 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7d10 (RPC#794) tag(31a002d) sent to node 1 [ CSSD]2013-05-23 17:45:41.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(031a002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:41.362 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:41.363 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7d10 (RPC#794) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:41.363 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7d10, ret 0, client 0x7f431c137890 member 0x7f431c137d00 [ CSSD]2013-05-23 17:45:41.363 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 794 [ CSSD]2013-05-23 17:45:41.363 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137d00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:45:41.534 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:41.534 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:41.565 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092560 [ CSSD]2013-05-23 17:45:41.565 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092560) con(0x7f431c091660) skgpid ospid 7664 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:42.423 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:42.754 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:42.754 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:43.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:43.540 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:44.477 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:45.010 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:45.010 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015448, gin 1, member 0 [ CSSD]2013-05-23 17:45:45.010 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:45:45.010 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:45:45.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:45.016 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:45:45.016 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:45:45.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:45.592 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:46.483 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:46.866 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:45:46.866 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 795 (0x7f431bfd7d90) [ CSSD]2013-05-23 17:45:46.867 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7d90 (RPC#795) tag(31b002d) sent to node 1 [ CSSD]2013-05-23 17:45:46.867 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:46.867 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 795 (0x7f431bfd7d90) [ CSSD]2013-05-23 17:45:46.867 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7d90 (RPC#795) state 6, flags 0x202 [ CSSD]2013-05-23 17:45:46.867 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7d90, ret 0, client (nil) [ CSSD]2013-05-23 17:45:46.867 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 795 [ CSSD]2013-05-23 17:45:47.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:47.598 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:47.757 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:47.757 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:48.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:49.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:49.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:50.278 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136fe0) allocated [ CSSD]2013-05-23 17:45:50.279 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:45:50.279 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136fe0 - 1,2,3 [ CSSD]2013-05-23 17:45:50.279 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090cf0) proc(0x7f431c136fe0) pid(7734) version 11:1:1:4 [ CSSD]2013-05-23 17:45:50.285 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c136fe0), client(1/0x7f431c136f40) [ CSSD]2013-05-23 17:45:50.285 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c136f40) [ CSSD]2013-05-23 17:45:50.285 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c136f40 with con 0x7f431c137270, requested num -1 [ CSSD]2013-05-23 17:45:50.285 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:50.285 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 796 (0x7f431bfd7e10) [ CSSD]2013-05-23 17:45:50.285 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7e10 (RPC#796) tag(31c002d) sent to node 1 [ CSSD]2013-05-23 17:45:50.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(031c002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:50.285 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015449, gin 1, member 0 [ CSSD]2013-05-23 17:45:50.285 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137860/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:45:50.286 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:45:50.286 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:45:50.286 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7e10 (RPC#796) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:50.286 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7e10, ret 0, client 0x7f431c136f40 member 0x7f431c137b60 [ CSSD]2013-05-23 17:45:50.286 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c136f40 [ CSSD]2013-05-23 17:45:50.286 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 796 [ CSSD]2013-05-23 17:45:50.292 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c136f40) [ CSSD]2013-05-23 17:45:50.292 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c136f40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:50.292 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137b60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:45:50.292 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 797 (0x7f431bfd7e90) [ CSSD]2013-05-23 17:45:50.292 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7e90 (RPC#797) tag(31d002d) sent to node 1 [ CSSD]2013-05-23 17:45:50.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(031d002d) incarnation(174631051) [ CSSD]2013-05-23 17:45:50.292 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:45:50.292 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7e90 (RPC#797) state 6, flags 0x100 [ CSSD]2013-05-23 17:45:50.292 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd7e90, ret 0, client 0x7f431c136f40 member 0x7f431c137b60 [ CSSD]2013-05-23 17:45:50.292 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 797 [ CSSD]2013-05-23 17:45:50.293 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137b60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:45:50.495 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136fe0 [ CSSD]2013-05-23 17:45:50.495 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136fe0) con(0x7f431c090cf0) skgpid ospid 7734 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:45:50.501 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:51.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:51.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:52.507 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:52.761 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:52.761 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:45:53.641 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:53.641 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:54.517 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:55.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:55.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:56.523 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:56.763 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:45:56.763 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:45:56.892 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:45:56.892 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 798 (0x7f431bfd7f10) [ CSSD]2013-05-23 17:45:56.893 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7f10 (RPC#798) tag(31e002d) sent to node 1 [ CSSD]2013-05-23 17:45:56.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:56.893 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 798 (0x7f431bfd7f10) [ CSSD]2013-05-23 17:45:56.893 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7f10 (RPC#798) state 6, flags 0x202 [ CSSD]2013-05-23 17:45:56.893 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd7f10, ret 0, client (nil) [ CSSD]2013-05-23 17:45:56.893 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 798 [ CSSD]2013-05-23 17:45:57.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:57.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:45:58.575 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:45:59.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:45:59.673 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:00.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:00.264 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015450, gin 1, member 0 [ CSSD]2013-05-23 17:46:00.265 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:46:00.265 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:46:00.272 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:00.272 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:46:00.272 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:46:00.620 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:01.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:01.724 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:01.766 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:01.766 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:02.626 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:03.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:03.730 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:04.678 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:05.526 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c092500) allocated [ CSSD]2013-05-23 17:46:05.527 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:46:05.528 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c092500 - 1,2,3 [ CSSD]2013-05-23 17:46:05.528 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091660) proc(0x7f431c092500) pid(7905) version 11:1:1:4 [ CSSD]2013-05-23 17:46:05.533 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c092500), client(1/0x7f431c092460) [ CSSD]2013-05-23 17:46:05.533 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c092460) [ CSSD]2013-05-23 17:46:05.533 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c092460 with con 0x7f431c090dc0, requested num -1 [ CSSD]2013-05-23 17:46:05.533 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:05.533 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 799 (0x7f431bfd7f90) [ CSSD]2013-05-23 17:46:05.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd7f90 (RPC#799) tag(31f002d) sent to node 1 [ CSSD]2013-05-23 17:46:05.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(031f002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:05.533 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015451, gin 1, member 0 [ CSSD]2013-05-23 17:46:05.533 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137a20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:46:05.533 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:46:05.534 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:46:05.534 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd7f90 (RPC#799) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:05.534 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd7f90, ret 0, client 0x7f431c092460 member 0x7f431c137d20 [ CSSD]2013-05-23 17:46:05.534 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c092460 [ CSSD]2013-05-23 17:46:05.534 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 799 [ CSSD]2013-05-23 17:46:05.539 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c092460) [ CSSD]2013-05-23 17:46:05.539 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c092460), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:05.539 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137d20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:05.539 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 800 (0x7f431bfd8010) [ CSSD]2013-05-23 17:46:05.540 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8010 (RPC#800) tag(320002d) sent to node 1 [ CSSD]2013-05-23 17:46:05.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0320002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:05.540 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:05.540 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8010 (RPC#800) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:05.540 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8010, ret 0, client 0x7f431c092460 member 0x7f431c137d20 [ CSSD]2013-05-23 17:46:05.540 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 800 [ CSSD]2013-05-23 17:46:05.540 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137d20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:46:05.742 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c092500 [ CSSD]2013-05-23 17:46:05.742 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c092500) con(0x7f431c091660) skgpid ospid 7905 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:05.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:05.743 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:06.684 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:06.769 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:06.769 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:06.917 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:46:06.917 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 801 (0x7f431bfd8090) [ CSSD]2013-05-23 17:46:06.917 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8090 (RPC#801) tag(321002d) sent to node 1 [ CSSD]2013-05-23 17:46:06.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:06.918 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 801 (0x7f431bfd8090) [ CSSD]2013-05-23 17:46:06.918 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8090 (RPC#801) state 6, flags 0x202 [ CSSD]2013-05-23 17:46:06.918 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8090, ret 0, client (nil) [ CSSD]2013-05-23 17:46:06.918 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 801 [ CSSD]2013-05-23 17:46:07.749 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:07.749 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:08.695 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:09.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:09.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:10.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:11.772 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:11.772 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:11.806 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:11.806 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:12.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:13.812 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:13.812 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:14.718 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:15.520 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:15.520 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015452, gin 1, member 0 [ CSSD]2013-05-23 17:46:15.520 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:46:15.520 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:46:15.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:15.528 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:46:15.528 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:46:15.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:15.823 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:16.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:16.774 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:16.774 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:16.945 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:46:16.945 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 802 (0x7f431bfd8110) [ CSSD]2013-05-23 17:46:16.945 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8110 (RPC#802) tag(322002d) sent to node 1 [ CSSD]2013-05-23 17:46:16.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:16.946 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 802 (0x7f431bfd8110) [ CSSD]2013-05-23 17:46:16.946 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8110 (RPC#802) state 6, flags 0x202 [ CSSD]2013-05-23 17:46:16.946 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8110, ret 0, client (nil) [ CSSD]2013-05-23 17:46:16.946 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 802 [ CSSD]2013-05-23 17:46:17.850 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:17.850 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:18.726 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:19.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:19.901 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:20.752 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:20.774 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1370b0) allocated [ CSSD]2013-05-23 17:46:20.775 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:46:20.775 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1370b0 - 1,2,3 [ CSSD]2013-05-23 17:46:20.775 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c091540) proc(0x7f431c1370b0) pid(8070) version 11:1:1:4 [ CSSD]2013-05-23 17:46:20.776 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:20.777 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:46:20.781 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1370b0), client(1/0x7f431c1379e0) [ CSSD]2013-05-23 17:46:20.781 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c1379e0) [ CSSD]2013-05-23 17:46:20.781 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c1379e0 with con 0x7f431c1374c0, requested num -1 [ CSSD]2013-05-23 17:46:20.781 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:20.781 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 803 (0x7f431bfd8190) [ CSSD]2013-05-23 17:46:20.781 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8190 (RPC#803) tag(323002d) sent to node 1 [ CSSD]2013-05-23 17:46:20.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0323002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:20.781 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015453, gin 1, member 0 [ CSSD]2013-05-23 17:46:20.781 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137b50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:46:20.782 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:46:20.782 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:46:20.782 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8190 (RPC#803) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:20.782 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8190, ret 0, client 0x7f431c1379e0 member 0x7f431c137340 [ CSSD]2013-05-23 17:46:20.782 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c1379e0 [ CSSD]2013-05-23 17:46:20.782 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 803 [ CSSD]2013-05-23 17:46:20.787 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c1379e0) [ CSSD]2013-05-23 17:46:20.787 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c1379e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:20.787 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137340) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:20.787 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 804 (0x7f431bfd8210) [ CSSD]2013-05-23 17:46:20.787 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8210 (RPC#804) tag(324002d) sent to node 1 [ CSSD]2013-05-23 17:46:20.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0324002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:20.788 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:20.788 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8210 (RPC#804) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:20.788 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8210, ret 0, client 0x7f431c1379e0 member 0x7f431c137340 [ CSSD]2013-05-23 17:46:20.788 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 804 [ CSSD]2013-05-23 17:46:20.788 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c137340) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:46:20.990 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1370b0 [ CSSD]2013-05-23 17:46:20.990 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1370b0) con(0x7f431c091540) skgpid ospid 8070 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:21.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:21.925 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:22.382 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c136fe0) allocated [ CSSD]2013-05-23 17:46:22.383 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:46:22.384 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c136fe0 - 1,2,3 [ CSSD]2013-05-23 17:46:22.384 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090cf0) proc(0x7f431c136fe0) pid(8135) version 11:1:1:4 [ CSSD]2013-05-23 17:46:22.391 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c136fe0), client(1/0x7f431c136f40) [ CSSD]2013-05-23 17:46:22.391 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c136f40) [ CSSD]2013-05-23 17:46:22.391 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c136f40 with con 0x7f431c137270, requested num -1 [ CSSD]2013-05-23 17:46:22.391 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:46:22.391 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 805 (0x7f431bfd8290) [ CSSD]2013-05-23 17:46:22.391 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8290 (RPC#805) tag(325002d) sent to node 1 [ CSSD]2013-05-23 17:46:22.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0325002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:22.391 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015454, gin 1, member 0 [ CSSD]2013-05-23 17:46:22.391 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137860/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:46:22.392 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:46:22.392 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:46:22.392 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8290 (RPC#805) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:22.392 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8290, ret 0, client 0x7f431c136f40 member 0x7f431c137b60 [ CSSD]2013-05-23 17:46:22.392 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c136f40 [ CSSD]2013-05-23 17:46:22.392 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 805 [ CSSD]2013-05-23 17:46:22.398 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c136f40) [ CSSD]2013-05-23 17:46:22.398 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c136f40), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:46:22.398 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c137b60) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:46:22.398 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 806 (0x7f431bfd8310) [ CSSD]2013-05-23 17:46:22.398 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8310 (RPC#806) tag(326002d) sent to node 1 [ CSSD]2013-05-23 17:46:22.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0326002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:22.398 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:46:22.398 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8310 (RPC#806) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:22.398 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8310, ret 0, client 0x7f431c136f40 member 0x7f431c137b60 [ CSSD]2013-05-23 17:46:22.398 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 806 [ CSSD]2013-05-23 17:46:22.398 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c137b60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:46:22.511 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bbf0) allocated [ CSSD]2013-05-23 17:46:22.512 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:46:22.512 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bbf0 - 1,2,3 [ CSSD]2013-05-23 17:46:22.512 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137800) proc(0x7f431c13bbf0) pid(8140) version 11:1:1:4 [ CSSD]2013-05-23 17:46:22.528 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:46:22.529 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:46:22.532 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bbf0 [ CSSD]2013-05-23 17:46:22.532 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bbf0) con(0x7f431c137800) skgpid ospid 8140 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:22.601 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c136fe0 [ CSSD]2013-05-23 17:46:22.601 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c136fe0) con(0x7f431c090cf0) skgpid ospid 8135 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:22.758 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:23.931 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:23.931 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:24.758 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:25.175 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22d7210) [ CSSD]2013-05-23 17:46:25.175 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22d7210), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:46:25.175 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22d7210), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:46:25.175 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 29 [ CSSD]2013-05-23 17:46:25.180 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22d7cd0) [ CSSD]2013-05-23 17:46:25.180 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22d7cd0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:46:25.180 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22d7cd0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:46:25.181 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 17:46:25.181 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:46:25.181 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2312880) skgpid 2382 ospid 2382 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:25.779 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:25.780 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:25.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:25.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:26.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:26.968 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:46:26.968 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 807 (0x7f431bfd8390) [ CSSD]2013-05-23 17:46:26.969 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8390 (RPC#807) tag(327002d) sent to node 1 [ CSSD]2013-05-23 17:46:26.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:26.969 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 807 (0x7f431bfd8390) [ CSSD]2013-05-23 17:46:26.969 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8390 (RPC#807) state 6, flags 0x202 [ CSSD]2013-05-23 17:46:26.969 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8390, ret 0, client (nil) [ CSSD]2013-05-23 17:46:26.970 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 807 [ CSSD]2013-05-23 17:46:27.950 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:27.950 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:28.816 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:30.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:30.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:30.782 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:30.782 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:30.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:30.791 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015455, gin 1, member 0 [ CSSD]2013-05-23 17:46:30.791 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:46:30.791 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:46:30.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:30.798 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:46:30.798 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:46:30.821 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:32.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:32.007 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:32.826 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:34.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:34.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:34.840 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:35.785 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:35.785 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:36.023 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1370a0) allocated [ CSSD]2013-05-23 17:46:36.023 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:46:36.024 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1370a0 - 1,2,3 [ CSSD]2013-05-23 17:46:36.024 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090cf0) proc(0x7f431c1370a0) pid(8257) version 11:1:1:4 [ CSSD]2013-05-23 17:46:36.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:36.025 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:36.029 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c1370a0), client(1/0x7f431c137790) [ CSSD]2013-05-23 17:46:36.029 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137790) [ CSSD]2013-05-23 17:46:36.029 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137790 with con 0x7f431c090da0, requested num -1 [ CSSD]2013-05-23 17:46:36.029 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:36.029 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 808 (0x7f431bfd8410) [ CSSD]2013-05-23 17:46:36.029 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8410 (RPC#808) tag(328002d) sent to node 1 [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0328002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015456, gin 1, member 0 [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c137900/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8410 (RPC#808) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8410, ret 0, client 0x7f431c137790 member 0x7f431c136f80 [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137790 [ CSSD]2013-05-23 17:46:36.030 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 808 [ CSSD]2013-05-23 17:46:36.035 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137790) [ CSSD]2013-05-23 17:46:36.035 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137790), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:36.035 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136f80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:36.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 809 (0x7f431bfd8490) [ CSSD]2013-05-23 17:46:36.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8490 (RPC#809) tag(329002d) sent to node 1 [ CSSD]2013-05-23 17:46:36.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0329002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:36.036 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:36.036 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8490 (RPC#809) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:36.036 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8490, ret 0, client 0x7f431c137790 member 0x7f431c136f80 [ CSSD]2013-05-23 17:46:36.036 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 809 [ CSSD]2013-05-23 17:46:36.036 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136f80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:46:36.238 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1370a0 [ CSSD]2013-05-23 17:46:36.238 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1370a0) con(0x7f431c090cf0) skgpid ospid 8257 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:36.845 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:36.998 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:46:36.998 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 810 (0x7f431bfd8510) [ CSSD]2013-05-23 17:46:36.998 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8510 (RPC#810) tag(32a002d) sent to node 1 [ CSSD]2013-05-23 17:46:36.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:36.999 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 810 (0x7f431bfd8510) [ CSSD]2013-05-23 17:46:36.999 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8510 (RPC#810) state 6, flags 0x202 [ CSSD]2013-05-23 17:46:36.999 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8510, ret 0, client (nil) [ CSSD]2013-05-23 17:46:36.999 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 810 [ CSSD]2013-05-23 17:46:37.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:37.721 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015457, gin 1, member 0 [ CSSD]2013-05-23 17:46:37.721 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:46:37.721 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:46:37.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:37.727 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:46:37.727 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:46:38.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:38.030 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:38.897 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:40.041 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:40.041 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:40.788 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:40.788 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:40.903 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:41.601 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1370a0) allocated [ CSSD]2013-05-23 17:46:41.602 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:46:41.602 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1370a0 - 1,2,3 [ CSSD]2013-05-23 17:46:41.602 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090cf0) proc(0x7f431c1370a0) pid(8335) version 11:1:1:4 [ CSSD]2013-05-23 17:46:41.608 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c1370a0), client(1/0x7f431c137840) [ CSSD]2013-05-23 17:46:41.608 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137840) [ CSSD]2013-05-23 17:46:41.608 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137840 with con 0x7f431c137330, requested num -1 [ CSSD]2013-05-23 17:46:41.608 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:41.608 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 811 (0x7f431bfd8590) [ CSSD]2013-05-23 17:46:41.609 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8590 (RPC#811) tag(32b002d) sent to node 1 [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(032b002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015458, gin 1, member 0 [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1379b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8590 (RPC#811) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8590, ret 0, client 0x7f431c137840 member 0x7f431c136f80 [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137840 [ CSSD]2013-05-23 17:46:41.609 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 811 [ CSSD]2013-05-23 17:46:41.616 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137840) [ CSSD]2013-05-23 17:46:41.616 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137840), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:41.616 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c136f80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:41.616 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 812 (0x7f431bfd8610) [ CSSD]2013-05-23 17:46:41.616 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8610 (RPC#812) tag(32c002d) sent to node 1 [ CSSD]2013-05-23 17:46:41.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(032c002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:41.616 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:41.616 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8610 (RPC#812) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:41.616 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8610, ret 0, client 0x7f431c137840 member 0x7f431c136f80 [ CSSD]2013-05-23 17:46:41.617 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 812 [ CSSD]2013-05-23 17:46:41.617 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c136f80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:46:41.819 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1370a0 [ CSSD]2013-05-23 17:46:41.819 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1370a0) con(0x7f431c090cf0) skgpid ospid 8335 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:42.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:42.047 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:42.909 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:44.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:44.074 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:44.791 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:44.791 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:46:44.939 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:46.040 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:46.040 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015459, gin 1, member 0 [ CSSD]2013-05-23 17:46:46.040 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:46:46.040 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:46:46.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:46.047 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:46:46.047 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:46:46.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:46.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:46.944 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:47.023 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:46:47.023 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 813 (0x7f431bfd8690) [ CSSD]2013-05-23 17:46:47.024 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8690 (RPC#813) tag(32d002d) sent to node 1 [ CSSD]2013-05-23 17:46:47.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:47.024 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 813 (0x7f431bfd8690) [ CSSD]2013-05-23 17:46:47.024 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8690 (RPC#813) state 6, flags 0x202 [ CSSD]2013-05-23 17:46:47.024 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8690, ret 0, client (nil) [ CSSD]2013-05-23 17:46:47.024 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 813 [ CSSD]2013-05-23 17:46:48.111 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:48.111 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:48.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:49.794 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:49.794 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:50.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:50.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:51.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:51.277 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1370a0) allocated [ CSSD]2013-05-23 17:46:51.278 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:46:51.278 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1370a0 - 1,2,3 [ CSSD]2013-05-23 17:46:51.278 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090cf0) proc(0x7f431c1370a0) pid(8442) version 11:1:1:4 [ CSSD]2013-05-23 17:46:51.284 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c1370a0), client(1/0x23191a0) [ CSSD]2013-05-23 17:46:51.284 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23191a0) [ CSSD]2013-05-23 17:46:51.284 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23191a0 with con 0x2302f30, requested num -1 [ CSSD]2013-05-23 17:46:51.284 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:51.284 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 814 (0x7f431bfd8710) [ CSSD]2013-05-23 17:46:51.284 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8710 (RPC#814) tag(32e002d) sent to node 1 [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(032e002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015460, gin 1, member 0 [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8710 (RPC#814) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8710, ret 0, client 0x23191a0 member 0x2302d30 [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23191a0 [ CSSD]2013-05-23 17:46:51.285 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 814 [ CSSD]2013-05-23 17:46:51.290 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23191a0) [ CSSD]2013-05-23 17:46:51.290 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23191a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:51.290 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2302d30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:46:51.291 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 815 (0x7f431bfd8790) [ CSSD]2013-05-23 17:46:51.291 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8790 (RPC#815) tag(32f002d) sent to node 1 [ CSSD]2013-05-23 17:46:51.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(032f002d) incarnation(174631051) [ CSSD]2013-05-23 17:46:51.291 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:46:51.291 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8790 (RPC#815) state 6, flags 0x100 [ CSSD]2013-05-23 17:46:51.291 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8790, ret 0, client 0x23191a0 member 0x2302d30 [ CSSD]2013-05-23 17:46:51.291 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 815 [ CSSD]2013-05-23 17:46:51.291 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2302d30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:46:51.494 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1370a0 [ CSSD]2013-05-23 17:46:51.494 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1370a0) con(0x7f431c090cf0) skgpid ospid 8442 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:46:52.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:52.146 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:53.010 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:54.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:54.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:54.797 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:54.797 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:46:55.020 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:56.205 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:56.205 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:57.045 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:57.052 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:46:57.052 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 816 (0x7f431bfd8810) [ CSSD]2013-05-23 17:46:57.053 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8810 (RPC#816) tag(330002d) sent to node 1 [ CSSD]2013-05-23 17:46:57.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:57.053 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 816 (0x7f431bfd8810) [ CSSD]2013-05-23 17:46:57.053 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8810 (RPC#816) state 6, flags 0x202 [ CSSD]2013-05-23 17:46:57.053 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8810, ret 0, client (nil) [ CSSD]2013-05-23 17:46:57.053 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 816 [ CSSD]2013-05-23 17:46:58.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:46:58.211 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:46:59.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:46:59.800 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:46:59.800 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:00.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:00.223 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:01.122 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:01.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:01.300 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015461, gin 1, member 0 [ CSSD]2013-05-23 17:47:01.300 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:01.300 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:01.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:01.308 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:01.308 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:02.248 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:02.248 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:03.128 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:03.802 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:03.802 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:47:04.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:04.254 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:05.141 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:06.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:06.306 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:06.524 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 17:47:06.525 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:47:06.526 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 17:47:06.526 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23191a0) proc(0x22d7cd0) pid(8649) version 11:1:1:4 [ CSSD]2013-05-23 17:47:06.543 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22d7cd0), client(1/0x2312950) [ CSSD]2013-05-23 17:47:06.543 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:47:06.543 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fc9c0, requested num -1 [ CSSD]2013-05-23 17:47:06.543 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:06.543 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 817 (0x7f431bfd8890) [ CSSD]2013-05-23 17:47:06.543 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8890 (RPC#817) tag(331002d) sent to node 1 [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0331002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015462, gin 1, member 0 [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8890 (RPC#817) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8890, ret 0, client 0x2312950 member 0x22dacd0 [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:47:06.543 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 817 [ CSSD]2013-05-23 17:47:06.550 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:47:06.550 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:06.550 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22dacd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:06.550 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 818 (0x7f431bfd8910) [ CSSD]2013-05-23 17:47:06.550 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8910 (RPC#818) tag(332002d) sent to node 1 [ CSSD]2013-05-23 17:47:06.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0332002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:06.550 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:06.550 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8910 (RPC#818) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:06.550 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8910, ret 0, client 0x2312950 member 0x22dacd0 [ CSSD]2013-05-23 17:47:06.550 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 818 [ CSSD]2013-05-23 17:47:06.550 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22dacd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:47:06.753 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 17:47:06.753 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x23191a0) skgpid ospid 8649 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:07.076 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:47:07.076 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 819 (0x7f431bfd8990) [ CSSD]2013-05-23 17:47:07.076 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8990 (RPC#819) tag(333002d) sent to node 1 [ CSSD]2013-05-23 17:47:07.077 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:07.077 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 819 (0x7f431bfd8990) [ CSSD]2013-05-23 17:47:07.077 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8990 (RPC#819) state 6, flags 0x202 [ CSSD]2013-05-23 17:47:07.077 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8990, ret 0, client (nil) [ CSSD]2013-05-23 17:47:07.077 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 819 [ CSSD]2013-05-23 17:47:07.145 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:08.312 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:08.312 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:08.805 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:08.805 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:09.199 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:10.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:10.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:11.203 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:12.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:12.329 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:13.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:13.808 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:13.808 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:14.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:14.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:14.971 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:14.971 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015463, gin 1, member 0 [ CSSD]2013-05-23 17:47:14.971 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:14.971 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:14.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:14.977 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:14.977 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:15.187 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c137120) allocated [ CSSD]2013-05-23 17:47:15.188 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:47:15.188 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c137120 - 1,2,3 [ CSSD]2013-05-23 17:47:15.188 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c090cf0) proc(0x7f431c137120) pid(8171/8171) version 11:1:1:4 [ CSSD]2013-05-23 17:47:15.192 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137120), client(1/0x7f431c1375c0) [ CSSD]2013-05-23 17:47:15.192 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c1375c0) [ CSSD]2013-05-23 17:47:15.192 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:47:15.192 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 17:47:15.221 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:16.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:16.425 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:16.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:16.557 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015464, gin 1, member 0 [ CSSD]2013-05-23 17:47:16.557 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:16.557 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:16.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:16.563 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:16.563 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:16.999 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c137120), client(2/0x7f431c137c40) [ CSSD]2013-05-23 17:47:16.999 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c137c40) [ CSSD]2013-05-23 17:47:16.999 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:47:16.999 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 30 [ CSSD]2013-05-23 17:47:17.151 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:47:17.152 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 820 (0x7f431bfd8a10) [ CSSD]2013-05-23 17:47:17.152 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8a10 (RPC#820) tag(334002d) sent to node 1 [ CSSD]2013-05-23 17:47:17.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:17.153 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 820 (0x7f431bfd8a10) [ CSSD]2013-05-23 17:47:17.153 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8a10 (RPC#820) state 6, flags 0x202 [ CSSD]2013-05-23 17:47:17.153 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8a10, ret 0, client (nil) [ CSSD]2013-05-23 17:47:17.153 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 820 [ CSSD]2013-05-23 17:47:17.226 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:18.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:18.429 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:18.811 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:18.811 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:19.239 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:20.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:20.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:21.174 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bd00) allocated [ CSSD]2013-05-23 17:47:21.175 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:47:21.176 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bd00 - 1,2,3 [ CSSD]2013-05-23 17:47:21.176 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13a3c0) proc(0x7f431c13bd00) pid(9085) version 11:1:1:4 [ CSSD]2013-05-23 17:47:21.246 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:21.295 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c141ec0) allocated [ CSSD]2013-05-23 17:47:21.296 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:47:21.297 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c141ec0 - 1,2,3 [ CSSD]2013-05-23 17:47:21.297 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14f150) proc(0x7f431c141ec0) pid(9095) version 11:1:1:4 [ CSSD]2013-05-23 17:47:21.312 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:47:21.314 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:47:21.316 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c141ec0 [ CSSD]2013-05-23 17:47:21.316 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c141ec0) con(0x7f431c14f150) skgpid ospid 9095 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:21.396 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(84/0x7f431c13bc60) [ CSSD]2013-05-23 17:47:21.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 84 (0x7f431c13bc60) [ CSSD]2013-05-23 17:47:21.396 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c13bc60 with con 0x7f431c137b00, requested num -1 [ CSSD]2013-05-23 17:47:21.396 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:47:21.396 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 821 (0x7f431bfd8a90) [ CSSD]2013-05-23 17:47:21.397 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8a90 (RPC#821) tag(335002d) sent to node 1 [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0335002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015465, gin 1, member 0 [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c14ef80/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8a90 (RPC#821) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8a90, ret 0, client 0x7f431c13bc60 member 0x7f431c14f280 [ CSSD]2013-05-23 17:47:21.397 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13bc60 [ CSSD]2013-05-23 17:47:21.398 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 821 [ CSSD]2013-05-23 17:47:21.403 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 84 (0x7f431c13bc60) [ CSSD]2013-05-23 17:47:21.403 [1191196992] >TRACE: clssgmExitGrock: client 84 (0x7f431c13bc60), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:47:21.403 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c14f280) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:47:21.403 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 822 (0x7f431bfd8b10) [ CSSD]2013-05-23 17:47:21.403 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8b10 (RPC#822) tag(336002d) sent to node 1 [ CSSD]2013-05-23 17:47:21.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0336002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:21.404 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:47:21.404 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8b10 (RPC#822) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:21.404 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8b10, ret 0, client 0x7f431c13bc60 member 0x7f431c14f280 [ CSSD]2013-05-23 17:47:21.404 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 822 [ CSSD]2013-05-23 17:47:21.404 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c14f280) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:47:21.405 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bd00 [ CSSD]2013-05-23 17:47:21.405 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bd00) con(0x7f431c13a3c0) skgpid ospid 9085 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:21.786 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bc50) allocated [ CSSD]2013-05-23 17:47:21.787 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:47:21.788 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bc50 - 1,2,3 [ CSSD]2013-05-23 17:47:21.788 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137470) proc(0x7f431c13bc50) pid(9105) version 11:1:1:4 [ CSSD]2013-05-23 17:47:21.793 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c13bc50), client(1/0x7f431c137520) [ CSSD]2013-05-23 17:47:21.793 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137520) [ CSSD]2013-05-23 17:47:21.793 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137520 with con 0x7f431c13bf20, requested num -1 [ CSSD]2013-05-23 17:47:21.793 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:21.793 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 823 (0x7f431bfd8b90) [ CSSD]2013-05-23 17:47:21.794 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8b90 (RPC#823) tag(337002d) sent to node 1 [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0337002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015466, gin 1, member 0 [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f1d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8b90 (RPC#823) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8b90, ret 0, client 0x7f431c137520 member 0x7f431c13bb20 [ CSSD]2013-05-23 17:47:21.794 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137520 [ CSSD]2013-05-23 17:47:21.795 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 823 [ CSSD]2013-05-23 17:47:21.800 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137520) [ CSSD]2013-05-23 17:47:21.800 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137520), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:21.800 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13bb20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:21.800 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 824 (0x7f431bfd8c10) [ CSSD]2013-05-23 17:47:21.800 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8c10 (RPC#824) tag(338002d) sent to node 1 [ CSSD]2013-05-23 17:47:21.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0338002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:21.801 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:21.801 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8c10 (RPC#824) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:21.801 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8c10, ret 0, client 0x7f431c137520 member 0x7f431c13bb20 [ CSSD]2013-05-23 17:47:21.801 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 824 [ CSSD]2013-05-23 17:47:21.801 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13bb20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:47:22.003 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bc50 [ CSSD]2013-05-23 17:47:22.003 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bc50) con(0x7f431c137470) skgpid ospid 9105 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:22.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:22.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:23.297 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:23.814 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:23.814 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:24.456 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:24.456 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:25.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:26.476 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:26.476 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:27.185 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:47:27.186 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 825 (0x7f431bfd8c90) [ CSSD]2013-05-23 17:47:27.186 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8c90 (RPC#825) tag(339002d) sent to node 1 [ CSSD]2013-05-23 17:47:27.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:27.186 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 825 (0x7f431bfd8c90) [ CSSD]2013-05-23 17:47:27.186 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8c90 (RPC#825) state 6, flags 0x202 [ CSSD]2013-05-23 17:47:27.186 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8c90, ret 0, client (nil) [ CSSD]2013-05-23 17:47:27.187 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 825 [ CSSD]2013-05-23 17:47:27.308 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:27.816 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:27.816 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:47:28.482 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:28.482 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:29.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:30.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:30.514 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:31.345 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:31.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:31.807 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015467, gin 1, member 0 [ CSSD]2013-05-23 17:47:31.807 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:31.807 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:31.813 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:31.813 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:31.813 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:32.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:32.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:32.819 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:32.820 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:33.397 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:34.544 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:34.544 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:35.443 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:36.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:36.558 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:37.044 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13be20) allocated [ CSSD]2013-05-23 17:47:37.045 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:47:37.046 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13be20 - 1,2,3 [ CSSD]2013-05-23 17:47:37.046 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13a480) proc(0x7f431c13be20) pid(9267) version 11:1:1:4 [ CSSD]2013-05-23 17:47:37.052 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c13be20), client(1/0x7f431c14f030) [ CSSD]2013-05-23 17:47:37.052 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14f030) [ CSSD]2013-05-23 17:47:37.052 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14f030 with con 0x7f431c13bd20, requested num -1 [ CSSD]2013-05-23 17:47:37.052 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:37.052 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 826 (0x7f431bfd8d10) [ CSSD]2013-05-23 17:47:37.052 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8d10 (RPC#826) tag(33a002d) sent to node 1 [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(033a002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015468, gin 1, member 0 [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f1a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8d10 (RPC#826) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8d10, ret 0, client 0x7f431c14f030 member 0x7f431c14f4a0 [ CSSD]2013-05-23 17:47:37.052 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14f030 [ CSSD]2013-05-23 17:47:37.053 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 826 [ CSSD]2013-05-23 17:47:37.059 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14f030) [ CSSD]2013-05-23 17:47:37.059 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14f030), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:37.059 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c14f4a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:37.059 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 827 (0x7f431bfd8d90) [ CSSD]2013-05-23 17:47:37.059 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8d90 (RPC#827) tag(33b002d) sent to node 1 [ CSSD]2013-05-23 17:47:37.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(033b002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:37.060 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:37.060 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8d90 (RPC#827) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:37.060 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8d90, ret 0, client 0x7f431c14f030 member 0x7f431c14f4a0 [ CSSD]2013-05-23 17:47:37.060 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 827 [ CSSD]2013-05-23 17:47:37.060 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c14f4a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:47:37.210 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:47:37.210 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 828 (0x7f431bfd8e10) [ CSSD]2013-05-23 17:47:37.211 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8e10 (RPC#828) tag(33c002d) sent to node 1 [ CSSD]2013-05-23 17:47:37.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:37.211 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 828 (0x7f431bfd8e10) [ CSSD]2013-05-23 17:47:37.211 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8e10 (RPC#828) state 6, flags 0x202 [ CSSD]2013-05-23 17:47:37.211 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8e10, ret 0, client (nil) [ CSSD]2013-05-23 17:47:37.211 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 828 [ CSSD]2013-05-23 17:47:37.262 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13be20 [ CSSD]2013-05-23 17:47:37.262 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13be20) con(0x7f431c13a480) skgpid ospid 9267 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:37.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:37.822 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:37.822 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:37.978 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:37.978 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015469, gin 1, member 0 [ CSSD]2013-05-23 17:47:37.978 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:37.978 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:37.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:37.984 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:37.984 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:38.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:38.564 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:39.450 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:40.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:40.615 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:41.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:41.864 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13be90) allocated [ CSSD]2013-05-23 17:47:41.865 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:47:41.866 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13be90 - 1,2,3 [ CSSD]2013-05-23 17:47:41.866 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13a550) proc(0x7f431c13be90) pid(9339) version 11:1:1:4 [ CSSD]2013-05-23 17:47:41.872 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c13be90), client(1/0x7f431c13bdf0) [ CSSD]2013-05-23 17:47:41.872 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13bdf0) [ CSSD]2013-05-23 17:47:41.872 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13bdf0 with con 0x7f431c13bfc0, requested num -1 [ CSSD]2013-05-23 17:47:41.872 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:41.872 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 829 (0x7f431bfd8e90) [ CSSD]2013-05-23 17:47:41.872 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8e90 (RPC#829) tag(33d002d) sent to node 1 [ CSSD]2013-05-23 17:47:41.872 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(033d002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:41.872 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015470, gin 1, member 0 [ CSSD]2013-05-23 17:47:41.872 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f1d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:47:41.873 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:47:41.873 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:47:41.873 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8e90 (RPC#829) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:41.873 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd8e90, ret 0, client 0x7f431c13bdf0 member 0x7f431c14f4d0 [ CSSD]2013-05-23 17:47:41.873 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13bdf0 [ CSSD]2013-05-23 17:47:41.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 829 [ CSSD]2013-05-23 17:47:41.879 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13bdf0) [ CSSD]2013-05-23 17:47:41.879 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13bdf0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:41.879 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c14f4d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:41.879 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 830 (0x7f431bfd8f10) [ CSSD]2013-05-23 17:47:41.879 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8f10 (RPC#830) tag(33e002d) sent to node 1 [ CSSD]2013-05-23 17:47:41.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(033e002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:41.879 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:41.879 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8f10 (RPC#830) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:41.879 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd8f10, ret 0, client 0x7f431c13bdf0 member 0x7f431c14f4d0 [ CSSD]2013-05-23 17:47:41.879 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 830 [ CSSD]2013-05-23 17:47:41.879 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c14f4d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:47:42.082 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13be90 [ CSSD]2013-05-23 17:47:42.082 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13be90) con(0x7f431c13a550) skgpid ospid 9339 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:42.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:42.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:42.825 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:42.825 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:43.508 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:44.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:44.497 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015471, gin 1, member 0 [ CSSD]2013-05-23 17:47:44.497 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:44.497 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:44.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:44.504 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:44.504 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:44.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:44.647 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:45.553 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:46.657 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:46.657 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:47.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:47.057 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015472, gin 1, member 0 [ CSSD]2013-05-23 17:47:47.057 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:47:47.057 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:47:47.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:47.064 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:47:47.064 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:47:47.234 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:47:47.234 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 831 (0x7f431bfd8f90) [ CSSD]2013-05-23 17:47:47.234 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd8f90 (RPC#831) tag(33f002d) sent to node 1 [ CSSD]2013-05-23 17:47:47.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:47.235 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 831 (0x7f431bfd8f90) [ CSSD]2013-05-23 17:47:47.235 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd8f90 (RPC#831) state 6, flags 0x202 [ CSSD]2013-05-23 17:47:47.235 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd8f90, ret 0, client (nil) [ CSSD]2013-05-23 17:47:47.235 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 831 [ CSSD]2013-05-23 17:47:47.559 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:47.828 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:47.828 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:48.663 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:48.663 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:49.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:50.577 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22ece50) [ CSSD]2013-05-23 17:47:50.577 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22ece50), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:47:50.577 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22ece50), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:47:50.577 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 29 [ CSSD]2013-05-23 17:47:50.587 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ec830) [ CSSD]2013-05-23 17:47:50.587 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ec830), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:47:50.587 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ec830), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:47:50.587 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 17:47:50.587 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f5000 [ CSSD]2013-05-23 17:47:50.587 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f5000) con(0x22f1f70) skgpid 3554 ospid 3554 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:50.716 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:50.716 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:51.617 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:51.830 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:51.830 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:47:52.293 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bdb0) allocated [ CSSD]2013-05-23 17:47:52.294 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:47:52.295 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bdb0 - 1,2,3 [ CSSD]2013-05-23 17:47:52.295 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13a480) proc(0x7f431c13bdb0) pid(9521) version 11:1:1:4 [ CSSD]2013-05-23 17:47:52.311 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c13bdb0), client(1/0x7f431c14f0f0) [ CSSD]2013-05-23 17:47:52.311 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14f0f0) [ CSSD]2013-05-23 17:47:52.311 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14f0f0 with con 0x7f431c137480, requested num -1 [ CSSD]2013-05-23 17:47:52.311 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:52.311 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 832 (0x7f431bfd9010) [ CSSD]2013-05-23 17:47:52.311 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9010 (RPC#832) tag(340002d) sent to node 1 [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0340002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015473, gin 1, member 0 [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f260/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9010 (RPC#832) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9010, ret 0, client 0x7f431c14f0f0 member 0x7f431c13bc90 [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14f0f0 [ CSSD]2013-05-23 17:47:52.311 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 832 [ CSSD]2013-05-23 17:47:52.317 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14f0f0) [ CSSD]2013-05-23 17:47:52.317 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14f0f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:52.317 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13bc90) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:47:52.317 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 833 (0x7f431bfd9090) [ CSSD]2013-05-23 17:47:52.318 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9090 (RPC#833) tag(341002d) sent to node 1 [ CSSD]2013-05-23 17:47:52.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0341002d) incarnation(174631051) [ CSSD]2013-05-23 17:47:52.318 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:47:52.318 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9090 (RPC#833) state 6, flags 0x100 [ CSSD]2013-05-23 17:47:52.318 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9090, ret 0, client 0x7f431c14f0f0 member 0x7f431c13bc90 [ CSSD]2013-05-23 17:47:52.318 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 833 [ CSSD]2013-05-23 17:47:52.318 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13bc90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:47:52.520 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bdb0 [ CSSD]2013-05-23 17:47:52.520 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bdb0) con(0x7f431c13a480) skgpid ospid 9521 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:47:52.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:52.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:53.628 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:54.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:54.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:55.635 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:56.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:56.758 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:56.833 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:47:56.833 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:47:57.257 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:47:57.257 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 834 (0x7f431bfd9110) [ CSSD]2013-05-23 17:47:57.258 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9110 (RPC#834) tag(342002d) sent to node 1 [ CSSD]2013-05-23 17:47:57.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:57.258 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 834 (0x7f431bfd9110) [ CSSD]2013-05-23 17:47:57.258 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9110 (RPC#834) state 6, flags 0x202 [ CSSD]2013-05-23 17:47:57.258 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9110, ret 0, client (nil) [ CSSD]2013-05-23 17:47:57.258 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 834 [ CSSD]2013-05-23 17:47:57.640 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:47:58.764 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:47:58.764 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:47:59.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:00.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:00.816 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:01.657 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:01.836 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:01.836 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:02.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:02.307 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015474, gin 1, member 0 [ CSSD]2013-05-23 17:48:02.307 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:48:02.307 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:48:02.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:02.314 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:48:02.314 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:48:02.842 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:02.842 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:03.709 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:04.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:04.854 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:05.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:06.839 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:06.839 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:06.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:06.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:07.282 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:48:07.282 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 835 (0x7f431bfd9190) [ CSSD]2013-05-23 17:48:07.282 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9190 (RPC#835) tag(343002d) sent to node 1 [ CSSD]2013-05-23 17:48:07.283 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:07.283 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 835 (0x7f431bfd9190) [ CSSD]2013-05-23 17:48:07.283 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9190 (RPC#835) state 6, flags 0x202 [ CSSD]2013-05-23 17:48:07.283 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9190, ret 0, client (nil) [ CSSD]2013-05-23 17:48:07.283 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 835 [ CSSD]2013-05-23 17:48:07.553 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bc40) allocated [ CSSD]2013-05-23 17:48:07.554 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:48:07.554 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bc40 - 1,2,3 [ CSSD]2013-05-23 17:48:07.554 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137af0) proc(0x7f431c13bc40) pid(9683) version 11:1:1:4 [ CSSD]2013-05-23 17:48:07.570 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c13bc40), client(1/0x7f431c137ba0) [ CSSD]2013-05-23 17:48:07.570 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c137ba0) [ CSSD]2013-05-23 17:48:07.570 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c137ba0 with con 0x7f431c13bf10, requested num -1 [ CSSD]2013-05-23 17:48:07.570 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:07.570 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 836 (0x7f431bfd9210) [ CSSD]2013-05-23 17:48:07.570 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9210 (RPC#836) tag(344002d) sent to node 1 [ CSSD]2013-05-23 17:48:07.570 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0344002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:07.570 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015475, gin 1, member 0 [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f100/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9210 (RPC#836) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9210, ret 0, client 0x7f431c137ba0 member 0x7f431c13bb20 [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c137ba0 [ CSSD]2013-05-23 17:48:07.571 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 836 [ CSSD]2013-05-23 17:48:07.577 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c137ba0) [ CSSD]2013-05-23 17:48:07.577 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c137ba0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:07.577 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13bb20) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:07.577 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 837 (0x7f431bfd9290) [ CSSD]2013-05-23 17:48:07.577 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9290 (RPC#837) tag(345002d) sent to node 1 [ CSSD]2013-05-23 17:48:07.577 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0345002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:07.577 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:07.577 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9290 (RPC#837) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:07.577 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9290, ret 0, client 0x7f431c137ba0 member 0x7f431c13bb20 [ CSSD]2013-05-23 17:48:07.578 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 837 [ CSSD]2013-05-23 17:48:07.578 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13bb20) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:48:07.720 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:07.780 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bc40 [ CSSD]2013-05-23 17:48:07.780 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bc40) con(0x7f431c137af0) skgpid ospid 9683 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:48:08.866 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:08.866 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:09.731 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:10.936 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:10.936 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:11.736 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:11.842 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:11.842 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:12.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:12.942 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:13.809 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:14.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:14.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:15.835 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:16.845 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:16.845 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:16.959 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:16.959 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:17.305 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:48:17.305 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 838 (0x7f431bfd9310) [ CSSD]2013-05-23 17:48:17.306 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9310 (RPC#838) tag(346002d) sent to node 1 [ CSSD]2013-05-23 17:48:17.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:17.306 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 838 (0x7f431bfd9310) [ CSSD]2013-05-23 17:48:17.306 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9310 (RPC#838) state 6, flags 0x202 [ CSSD]2013-05-23 17:48:17.306 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9310, ret 0, client (nil) [ CSSD]2013-05-23 17:48:17.306 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 838 [ CSSD]2013-05-23 17:48:17.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:17.568 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015476, gin 1, member 0 [ CSSD]2013-05-23 17:48:17.568 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:48:17.568 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:48:17.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:17.574 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:48:17.574 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:48:17.839 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:18.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:18.985 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:19.851 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:21.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:21.036 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:21.848 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:21.848 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:21.856 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:22.812 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bcf0) allocated [ CSSD]2013-05-23 17:48:22.813 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:48:22.814 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bcf0 - 1,2,3 [ CSSD]2013-05-23 17:48:22.814 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13a3c0) proc(0x7f431c13bcf0) pid(9860) version 11:1:1:4 [ CSSD]2013-05-23 17:48:22.819 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c13bcf0), client(1/0x7f431c14f0f0) [ CSSD]2013-05-23 17:48:22.819 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14f0f0) [ CSSD]2013-05-23 17:48:22.819 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c14f0f0 with con 0x7f431c13bf80, requested num -1 [ CSSD]2013-05-23 17:48:22.819 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:22.819 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 839 (0x7f431bfd9390) [ CSSD]2013-05-23 17:48:22.819 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9390 (RPC#839) tag(347002d) sent to node 1 [ CSSD]2013-05-23 17:48:22.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0347002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015477, gin 1, member 0 [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f260/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9390 (RPC#839) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9390, ret 0, client 0x7f431c14f0f0 member 0x7f431c13bbd0 [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14f0f0 [ CSSD]2013-05-23 17:48:22.820 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 839 [ CSSD]2013-05-23 17:48:22.825 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14f0f0) [ CSSD]2013-05-23 17:48:22.825 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14f0f0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:22.825 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13bbd0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:22.825 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 840 (0x7f431bfd9410) [ CSSD]2013-05-23 17:48:22.825 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9410 (RPC#840) tag(348002d) sent to node 1 [ CSSD]2013-05-23 17:48:22.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0348002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:22.826 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:22.826 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9410 (RPC#840) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:22.826 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9410, ret 0, client 0x7f431c14f0f0 member 0x7f431c13bbd0 [ CSSD]2013-05-23 17:48:22.826 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 840 [ CSSD]2013-05-23 17:48:22.826 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13bbd0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:48:23.028 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bcf0 [ CSSD]2013-05-23 17:48:23.028 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bcf0) con(0x7f431c13a3c0) skgpid ospid 9860 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:48:23.042 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:23.042 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:23.910 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:25.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:25.054 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:25.916 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:26.850 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:26.850 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:27.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:27.060 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:27.328 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:48:27.329 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 841 (0x7f431bfd9490) [ CSSD]2013-05-23 17:48:27.329 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9490 (RPC#841) tag(349002d) sent to node 1 [ CSSD]2013-05-23 17:48:27.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:27.330 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 841 (0x7f431bfd9490) [ CSSD]2013-05-23 17:48:27.330 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9490 (RPC#841) state 6, flags 0x202 [ CSSD]2013-05-23 17:48:27.330 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9490, ret 0, client (nil) [ CSSD]2013-05-23 17:48:27.330 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 841 [ CSSD]2013-05-23 17:48:27.940 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:29.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:29.085 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:29.951 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:31.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:31.137 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:31.209 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13bc40) allocated [ CSSD]2013-05-23 17:48:31.210 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:48:31.210 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13bc40 - 1,2,3 [ CSSD]2013-05-23 17:48:31.210 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c137470) proc(0x7f431c13bc40) pid(9500/9500) version 11:1:1:4 [ CSSD]2013-05-23 17:48:31.226 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c13bc40), client(1/0x7f431c137520) [ CSSD]2013-05-23 17:48:31.226 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c137520) [ CSSD]2013-05-23 17:48:31.226 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:48:31.226 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 17:48:31.853 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:31.853 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:31.957 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:32.829 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:32.829 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015478, gin 1, member 0 [ CSSD]2013-05-23 17:48:32.829 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:48:32.829 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:48:32.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:32.835 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:48:32.835 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:48:33.143 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:33.143 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:34.010 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:34.843 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c13bc40), client(2/0x7f431c14f3d0) [ CSSD]2013-05-23 17:48:34.843 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c14f3d0) [ CSSD]2013-05-23 17:48:34.843 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:48:34.843 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 30 [ CSSD]2013-05-23 17:48:35.155 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:35.155 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:35.856 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:35.856 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:48:36.016 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:37.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:37.159 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:37.354 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:48:37.355 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 842 (0x7f431bfd9510) [ CSSD]2013-05-23 17:48:37.355 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9510 (RPC#842) tag(34a002d) sent to node 1 [ CSSD]2013-05-23 17:48:37.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:37.355 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 842 (0x7f431bfd9510) [ CSSD]2013-05-23 17:48:37.355 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9510 (RPC#842) state 6, flags 0x202 [ CSSD]2013-05-23 17:48:37.355 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9510, ret 0, client (nil) [ CSSD]2013-05-23 17:48:37.356 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 842 [ CSSD]2013-05-23 17:48:38.022 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:38.059 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1428e0) allocated [ CSSD]2013-05-23 17:48:38.060 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:48:38.061 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1428e0 - 1,2,3 [ CSSD]2013-05-23 17:48:38.061 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14fbe0) proc(0x7f431c1428e0) pid(10022) version 11:1:1:4 [ CSSD]2013-05-23 17:48:38.067 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1428e0), client(1/0x7f431c142840) [ CSSD]2013-05-23 17:48:38.067 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142840) [ CSSD]2013-05-23 17:48:38.067 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142840 with con 0x7f431c142b90, requested num -1 [ CSSD]2013-05-23 17:48:38.067 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:38.067 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 843 (0x7f431bfd9590) [ CSSD]2013-05-23 17:48:38.067 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9590 (RPC#843) tag(34b002d) sent to node 1 [ CSSD]2013-05-23 17:48:38.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(034b002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:38.067 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015479, gin 1, member 0 [ CSSD]2013-05-23 17:48:38.067 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c143180/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:48:38.068 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:48:38.068 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:48:38.068 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9590 (RPC#843) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:38.068 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9590, ret 0, client 0x7f431c142840 member 0x7f431c143480 [ CSSD]2013-05-23 17:48:38.068 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142840 [ CSSD]2013-05-23 17:48:38.068 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 843 [ CSSD]2013-05-23 17:48:38.073 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142840) [ CSSD]2013-05-23 17:48:38.073 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142840), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:38.073 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c143480) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:38.073 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 844 (0x7f431bfd9610) [ CSSD]2013-05-23 17:48:38.073 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9610 (RPC#844) tag(34c002d) sent to node 1 [ CSSD]2013-05-23 17:48:38.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(034c002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:38.074 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:38.074 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9610 (RPC#844) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:38.074 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9610, ret 0, client 0x7f431c142840 member 0x7f431c143480 [ CSSD]2013-05-23 17:48:38.074 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 844 [ CSSD]2013-05-23 17:48:38.074 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c143480) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:48:38.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:38.244 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015480, gin 1, member 0 [ CSSD]2013-05-23 17:48:38.245 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:48:38.245 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:48:38.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:38.250 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:48:38.250 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:48:38.276 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1428e0 [ CSSD]2013-05-23 17:48:38.276 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1428e0) con(0x7f431c14fbe0) skgpid ospid 10022 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:48:39.182 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:39.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:40.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:40.858 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:40.858 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:41.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:41.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:42.059 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:42.117 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c142810) allocated [ CSSD]2013-05-23 17:48:42.119 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:48:42.119 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c142810 - 1,2,3 [ CSSD]2013-05-23 17:48:42.119 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14fab0) proc(0x7f431c142810) pid(10088) version 11:1:1:4 [ CSSD]2013-05-23 17:48:42.125 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c142810), client(1/0x7f431c142770) [ CSSD]2013-05-23 17:48:42.125 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:48:42.125 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142770 with con 0x7f431c142ae0, requested num -1 [ CSSD]2013-05-23 17:48:42.125 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:42.125 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 845 (0x7f431bfd9690) [ CSSD]2013-05-23 17:48:42.125 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9690 (RPC#845) tag(34d002d) sent to node 1 [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(034d002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015481, gin 1, member 0 [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1430d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9690 (RPC#845) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9690, ret 0, client 0x7f431c142770 member 0x7f431c1433d0 [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142770 [ CSSD]2013-05-23 17:48:42.126 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 845 [ CSSD]2013-05-23 17:48:42.132 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:48:42.132 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142770), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:42.132 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1433d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:42.132 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 846 (0x7f431bfd9710) [ CSSD]2013-05-23 17:48:42.132 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9710 (RPC#846) tag(34e002d) sent to node 1 [ CSSD]2013-05-23 17:48:42.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(034e002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:42.132 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:42.132 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9710 (RPC#846) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:42.132 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9710, ret 0, client 0x7f431c142770 member 0x7f431c1433d0 [ CSSD]2013-05-23 17:48:42.132 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 846 [ CSSD]2013-05-23 17:48:42.132 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1433d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:48:42.335 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c142810 [ CSSD]2013-05-23 17:48:42.335 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c142810) con(0x7f431c14fab0) skgpid ospid 10088 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:48:43.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:43.232 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:43.756 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:43.756 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015482, gin 1, member 0 [ CSSD]2013-05-23 17:48:43.756 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:48:43.756 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:48:43.763 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:43.763 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:48:43.763 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:48:43.890 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:48:43.891 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:48:43.892 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:48:43.893 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:48:43.893 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:48:43.894 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:48:44.111 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:45.326 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:45.326 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:45.861 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:45.862 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:46.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:47.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:47.352 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:47.382 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:48:47.383 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 847 (0x7f431bfd9790) [ CSSD]2013-05-23 17:48:47.383 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9790 (RPC#847) tag(34f002d) sent to node 1 [ CSSD]2013-05-23 17:48:47.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:47.383 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 847 (0x7f431bfd9790) [ CSSD]2013-05-23 17:48:47.383 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9790 (RPC#847) state 6, flags 0x202 [ CSSD]2013-05-23 17:48:47.383 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9790, ret 0, client (nil) [ CSSD]2013-05-23 17:48:47.383 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 847 [ CSSD]2013-05-23 17:48:48.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:48.075 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015483, gin 1, member 0 [ CSSD]2013-05-23 17:48:48.075 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:48:48.075 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:48:48.083 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:48.083 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:48:48.083 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:48:48.142 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:49.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:49.358 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:50.154 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:50.865 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:50.865 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:51.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:51.368 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:52.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:53.307 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1428e0) allocated [ CSSD]2013-05-23 17:48:53.308 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:48:53.309 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1428e0 - 1,2,3 [ CSSD]2013-05-23 17:48:53.309 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14fbd0) proc(0x7f431c1428e0) pid(10181) version 11:1:1:4 [ CSSD]2013-05-23 17:48:53.314 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1428e0), client(1/0x7f431c142840) [ CSSD]2013-05-23 17:48:53.314 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142840) [ CSSD]2013-05-23 17:48:53.314 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142840 with con 0x7f431c14f290, requested num -1 [ CSSD]2013-05-23 17:48:53.314 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:53.314 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 848 (0x7f431bfd9810) [ CSSD]2013-05-23 17:48:53.314 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9810 (RPC#848) tag(350002d) sent to node 1 [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0350002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015484, gin 1, member 0 [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1430f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9810 (RPC#848) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9810, ret 0, client 0x7f431c142840 member 0x7f431c1433f0 [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142840 [ CSSD]2013-05-23 17:48:53.315 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 848 [ CSSD]2013-05-23 17:48:53.320 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142840) [ CSSD]2013-05-23 17:48:53.320 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142840), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:53.320 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1433f0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:48:53.320 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 849 (0x7f431bfd9890) [ CSSD]2013-05-23 17:48:53.321 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9890 (RPC#849) tag(351002d) sent to node 1 [ CSSD]2013-05-23 17:48:53.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0351002d) incarnation(174631051) [ CSSD]2013-05-23 17:48:53.321 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:48:53.321 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9890 (RPC#849) state 6, flags 0x100 [ CSSD]2013-05-23 17:48:53.321 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9890, ret 0, client 0x7f431c142840 member 0x7f431c1433f0 [ CSSD]2013-05-23 17:48:53.321 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 849 [ CSSD]2013-05-23 17:48:53.321 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1433f0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:48:53.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:53.374 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:53.523 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1428e0 [ CSSD]2013-05-23 17:48:53.523 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1428e0) con(0x7f431c14fbd0) skgpid ospid 10181 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:48:54.211 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:55.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:55.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:55.868 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:55.868 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:48:56.217 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:57.408 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:48:57.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 850 (0x7f431bfd9910) [ CSSD]2013-05-23 17:48:57.408 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9910 (RPC#850) tag(352002d) sent to node 1 [ CSSD]2013-05-23 17:48:57.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:57.409 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 850 (0x7f431bfd9910) [ CSSD]2013-05-23 17:48:57.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9910 (RPC#850) state 6, flags 0x202 [ CSSD]2013-05-23 17:48:57.409 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9910, ret 0, client (nil) [ CSSD]2013-05-23 17:48:57.409 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 850 [ CSSD]2013-05-23 17:48:57.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:57.453 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:58.223 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:48:59.459 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:48:59.459 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:48:59.871 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:48:59.871 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:49:00.243 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:01.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:01.469 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:02.249 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:03.325 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:03.325 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015485, gin 1, member 0 [ CSSD]2013-05-23 17:49:03.325 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:49:03.325 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:49:03.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:03.333 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:49:03.333 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:49:03.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:03.475 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:04.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:04.874 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:04.874 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:05.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:05.527 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:06.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:07.440 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:49:07.440 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 851 (0x7f431bfd9990) [ CSSD]2013-05-23 17:49:07.441 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9990 (RPC#851) tag(353002d) sent to node 1 [ CSSD]2013-05-23 17:49:07.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:07.441 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 851 (0x7f431bfd9990) [ CSSD]2013-05-23 17:49:07.441 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9990 (RPC#851) state 6, flags 0x202 [ CSSD]2013-05-23 17:49:07.441 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9990, ret 0, client (nil) [ CSSD]2013-05-23 17:49:07.441 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 851 [ CSSD]2013-05-23 17:49:07.552 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:07.552 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:08.290 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:08.555 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c142810) allocated [ CSSD]2013-05-23 17:49:08.555 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:49:08.556 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c142810 - 1,2,3 [ CSSD]2013-05-23 17:49:08.556 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14fab0) proc(0x7f431c142810) pid(10347) version 11:1:1:4 [ CSSD]2013-05-23 17:49:08.561 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c142810), client(1/0x7f431c142770) [ CSSD]2013-05-23 17:49:08.561 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:08.561 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142770 with con 0x7f431c142ac0, requested num -1 [ CSSD]2013-05-23 17:49:08.561 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:08.561 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 852 (0x7f431bfd9a10) [ CSSD]2013-05-23 17:49:08.562 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9a10 (RPC#852) tag(354002d) sent to node 1 [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0354002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015486, gin 1, member 0 [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1430b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9a10 (RPC#852) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9a10, ret 0, client 0x7f431c142770 member 0x7f431c1433b0 [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142770 [ CSSD]2013-05-23 17:49:08.562 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 852 [ CSSD]2013-05-23 17:49:08.568 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:08.568 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142770), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:08.568 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1433b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:08.568 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 853 (0x7f431bfd9a90) [ CSSD]2013-05-23 17:49:08.568 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9a90 (RPC#853) tag(355002d) sent to node 1 [ CSSD]2013-05-23 17:49:08.569 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0355002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:08.569 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:08.569 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9a90 (RPC#853) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:08.569 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9a90, ret 0, client 0x7f431c142770 member 0x7f431c1433b0 [ CSSD]2013-05-23 17:49:08.569 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 853 [ CSSD]2013-05-23 17:49:08.569 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1433b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:49:08.772 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c142810 [ CSSD]2013-05-23 17:49:08.772 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c142810) con(0x7f431c14fab0) skgpid ospid 10347 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:49:09.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:09.557 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:09.877 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:09.877 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:10.296 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:11.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:11.589 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:12.302 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:13.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:13.595 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:14.333 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:14.880 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:14.881 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:15.627 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:15.627 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:16.339 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:17.465 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:49:17.465 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 854 (0x7f431bfd9b10) [ CSSD]2013-05-23 17:49:17.465 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9b10 (RPC#854) tag(356002d) sent to node 1 [ CSSD]2013-05-23 17:49:17.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:17.466 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 854 (0x7f431bfd9b10) [ CSSD]2013-05-23 17:49:17.466 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9b10 (RPC#854) state 6, flags 0x202 [ CSSD]2013-05-23 17:49:17.466 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9b10, ret 0, client (nil) [ CSSD]2013-05-23 17:49:17.466 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 854 [ CSSD]2013-05-23 17:49:17.633 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:17.633 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:18.350 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:18.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:18.585 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015487, gin 1, member 0 [ CSSD]2013-05-23 17:49:18.585 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:49:18.585 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:49:18.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:18.592 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:49:18.592 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:49:18.883 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:18.883 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:49:19.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:19.658 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:20.356 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:21.670 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:21.670 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:22.362 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:23.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:23.676 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:23.803 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c142810) allocated [ CSSD]2013-05-23 17:49:23.804 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:49:23.805 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c142810 - 1,2,3 [ CSSD]2013-05-23 17:49:23.805 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14ed40) proc(0x7f431c142810) pid(10520) version 11:1:1:4 [ CSSD]2013-05-23 17:49:23.820 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c142810), client(1/0x7f431c142770) [ CSSD]2013-05-23 17:49:23.820 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:23.820 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142770 with con 0x7f431c14edf0, requested num -1 [ CSSD]2013-05-23 17:49:23.820 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:23.820 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 855 (0x7f431bfd9b90) [ CSSD]2013-05-23 17:49:23.821 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9b90 (RPC#855) tag(357002d) sent to node 1 [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0357002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015488, gin 1, member 0 [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c142f70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9b90 (RPC#855) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9b90, ret 0, client 0x7f431c142770 member 0x7f431c143270 [ CSSD]2013-05-23 17:49:23.821 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142770 [ CSSD]2013-05-23 17:49:23.822 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 855 [ CSSD]2013-05-23 17:49:23.828 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:23.828 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142770), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:23.828 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c143270) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:23.828 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 856 (0x7f431bfd9c10) [ CSSD]2013-05-23 17:49:23.828 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9c10 (RPC#856) tag(358002d) sent to node 1 [ CSSD]2013-05-23 17:49:23.828 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0358002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:23.828 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:23.828 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9c10 (RPC#856) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:23.828 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9c10, ret 0, client 0x7f431c142770 member 0x7f431c143270 [ CSSD]2013-05-23 17:49:23.829 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 856 [ CSSD]2013-05-23 17:49:23.829 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c143270) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:49:23.886 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:23.887 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:24.031 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c142810 [ CSSD]2013-05-23 17:49:24.031 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c142810) con(0x7f431c14ed40) skgpid ospid 10520 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:49:24.373 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:25.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:25.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:26.379 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:27.489 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:49:27.489 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 857 (0x7f431bfd9c90) [ CSSD]2013-05-23 17:49:27.490 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9c90 (RPC#857) tag(359002d) sent to node 1 [ CSSD]2013-05-23 17:49:27.490 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:27.490 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 857 (0x7f431bfd9c90) [ CSSD]2013-05-23 17:49:27.490 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9c90 (RPC#857) state 6, flags 0x202 [ CSSD]2013-05-23 17:49:27.490 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9c90, ret 0, client (nil) [ CSSD]2013-05-23 17:49:27.490 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 857 [ CSSD]2013-05-23 17:49:27.754 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:27.754 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:28.431 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:28.889 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:28.890 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:29.826 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:29.826 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:30.437 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:31.082 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:49:31.083 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:49:31.084 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:49:31.084 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:49:31.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:31.168 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015489, gin 1, member 0 [ CSSD]2013-05-23 17:49:31.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:49:31.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:49:31.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:31.175 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:49:31.175 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:49:31.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:31.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:32.463 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:33.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:33.836 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015490, gin 1, member 0 [ CSSD]2013-05-23 17:49:33.836 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:49:33.836 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:49:33.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:33.844 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:49:33.844 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:49:33.857 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:33.857 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:33.893 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:33.893 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:34.472 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:35.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:35.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:36.478 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:37.518 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:49:37.518 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 858 (0x7f431bfd9d10) [ CSSD]2013-05-23 17:49:37.518 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9d10 (RPC#858) tag(35a002d) sent to node 1 [ CSSD]2013-05-23 17:49:37.518 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:37.519 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 858 (0x7f431bfd9d10) [ CSSD]2013-05-23 17:49:37.519 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9d10 (RPC#858) state 6, flags 0x202 [ CSSD]2013-05-23 17:49:37.519 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9d10, ret 0, client (nil) [ CSSD]2013-05-23 17:49:37.519 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 858 [ CSSD]2013-05-23 17:49:37.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:37.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:38.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:38.506 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015491, gin 1, member 0 [ CSSD]2013-05-23 17:49:38.506 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:49:38.506 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:49:38.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:38.513 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:49:38.513 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:49:38.531 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:38.896 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:38.896 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:39.076 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1429f0) allocated [ CSSD]2013-05-23 17:49:39.077 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:49:39.078 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1429f0 - 1,2,3 [ CSSD]2013-05-23 17:49:39.078 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14fca0) proc(0x7f431c1429f0) pid(10667) version 11:1:1:4 [ CSSD]2013-05-23 17:49:39.095 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1429f0), client(1/0x7f431c142950) [ CSSD]2013-05-23 17:49:39.095 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142950) [ CSSD]2013-05-23 17:49:39.095 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142950 with con 0x7f431c14f2d0, requested num -1 [ CSSD]2013-05-23 17:49:39.095 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:39.095 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 859 (0x7f431bfd9d90) [ CSSD]2013-05-23 17:49:39.095 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9d90 (RPC#859) tag(35b002d) sent to node 1 [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(035b002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015492, gin 1, member 0 [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c143190/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9d90 (RPC#859) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9d90, ret 0, client 0x7f431c142950 member 0x7f431c143490 [ CSSD]2013-05-23 17:49:39.095 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142950 [ CSSD]2013-05-23 17:49:39.096 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 859 [ CSSD]2013-05-23 17:49:39.103 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142950) [ CSSD]2013-05-23 17:49:39.103 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:39.103 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c143490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:39.103 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 860 (0x7f431bfd9e10) [ CSSD]2013-05-23 17:49:39.104 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9e10 (RPC#860) tag(35c002d) sent to node 1 [ CSSD]2013-05-23 17:49:39.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(035c002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:39.104 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:39.104 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9e10 (RPC#860) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:39.104 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9e10, ret 0, client 0x7f431c142950 member 0x7f431c143490 [ CSSD]2013-05-23 17:49:39.104 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 860 [ CSSD]2013-05-23 17:49:39.104 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c143490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:49:39.307 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1429f0 [ CSSD]2013-05-23 17:49:39.307 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1429f0) con(0x7f431c14fca0) skgpid ospid 10667 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:49:39.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:39.926 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:40.537 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:41.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:41.933 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:42.378 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c142810) allocated [ CSSD]2013-05-23 17:49:42.379 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:49:42.380 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c142810 - 1,2,3 [ CSSD]2013-05-23 17:49:42.380 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14ed40) proc(0x7f431c142810) pid(10727) version 11:1:1:4 [ CSSD]2013-05-23 17:49:42.396 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c142810), client(1/0x7f431c142770) [ CSSD]2013-05-23 17:49:42.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:42.396 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142770 with con 0x7f431c14edf0, requested num -1 [ CSSD]2013-05-23 17:49:42.396 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:42.396 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 861 (0x7f431bfd9e90) [ CSSD]2013-05-23 17:49:42.396 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9e90 (RPC#861) tag(35d002d) sent to node 1 [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(035d002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015493, gin 1, member 0 [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c142f70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9e90 (RPC#861) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfd9e90, ret 0, client 0x7f431c142770 member 0x7f431c143270 [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142770 [ CSSD]2013-05-23 17:49:42.397 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 861 [ CSSD]2013-05-23 17:49:42.403 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:42.403 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142770), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:42.403 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c143270) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:42.404 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 862 (0x7f431bfd9f10) [ CSSD]2013-05-23 17:49:42.404 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9f10 (RPC#862) tag(35e002d) sent to node 1 [ CSSD]2013-05-23 17:49:42.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(035e002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:42.404 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:42.404 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9f10 (RPC#862) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:42.404 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfd9f10, ret 0, client 0x7f431c142770 member 0x7f431c143270 [ CSSD]2013-05-23 17:49:42.404 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 862 [ CSSD]2013-05-23 17:49:42.404 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c143270) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:49:42.562 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:42.607 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c142810 [ CSSD]2013-05-23 17:49:42.607 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c142810) con(0x7f431c14ed40) skgpid ospid 10727 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:49:43.899 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:43.899 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:43.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:43.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:44.592 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:45.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:45.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:46.597 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:47.542 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:49:47.542 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 863 (0x7f431bfd9f90) [ CSSD]2013-05-23 17:49:47.543 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfd9f90 (RPC#863) tag(35f002d) sent to node 1 [ CSSD]2013-05-23 17:49:47.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:47.543 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 863 (0x7f431bfd9f90) [ CSSD]2013-05-23 17:49:47.543 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfd9f90 (RPC#863) state 6, flags 0x202 [ CSSD]2013-05-23 17:49:47.543 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfd9f90, ret 0, client (nil) [ CSSD]2013-05-23 17:49:47.543 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 863 [ CSSD]2013-05-23 17:49:47.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:47.975 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:48.631 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:48.902 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:48.902 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:49.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:49.085 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015494, gin 1, member 0 [ CSSD]2013-05-23 17:49:49.085 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:49:49.085 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:49:49.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:49.092 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:49:49.092 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:49:50.027 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:50.027 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:50.637 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:52.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:52.033 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:52.643 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:53.905 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:53.905 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:49:54.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:54.038 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:54.338 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c142810) allocated [ CSSD]2013-05-23 17:49:54.339 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:49:54.339 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c142810 - 1,2,3 [ CSSD]2013-05-23 17:49:54.339 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14ed40) proc(0x7f431c142810) pid(10845) version 11:1:1:4 [ CSSD]2013-05-23 17:49:54.344 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c142810), client(1/0x7f431c142770) [ CSSD]2013-05-23 17:49:54.344 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:54.344 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142770 with con 0x7f431c14edf0, requested num -1 [ CSSD]2013-05-23 17:49:54.344 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:54.344 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 864 (0x7f431bfda010) [ CSSD]2013-05-23 17:49:54.344 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda010 (RPC#864) tag(360002d) sent to node 1 [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0360002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015495, gin 1, member 0 [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c142f70/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda010 (RPC#864) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda010, ret 0, client 0x7f431c142770 member 0x7f431c143270 [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142770 [ CSSD]2013-05-23 17:49:54.345 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 864 [ CSSD]2013-05-23 17:49:54.350 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:49:54.350 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142770), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:54.350 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c143270) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:49:54.350 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 865 (0x7f431bfda090) [ CSSD]2013-05-23 17:49:54.350 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda090 (RPC#865) tag(361002d) sent to node 1 [ CSSD]2013-05-23 17:49:54.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0361002d) incarnation(174631051) [ CSSD]2013-05-23 17:49:54.351 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:49:54.351 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda090 (RPC#865) state 6, flags 0x100 [ CSSD]2013-05-23 17:49:54.351 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda090, ret 0, client 0x7f431c142770 member 0x7f431c143270 [ CSSD]2013-05-23 17:49:54.351 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 865 [ CSSD]2013-05-23 17:49:54.351 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c143270) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:49:54.553 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c142810 [ CSSD]2013-05-23 17:49:54.553 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c142810) con(0x7f431c14ed40) skgpid ospid 10845 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:49:54.655 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:56.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:56.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:56.661 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:57.567 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:49:57.567 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 866 (0x7f431bfda110) [ CSSD]2013-05-23 17:49:57.567 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda110 (RPC#866) tag(362002d) sent to node 1 [ CSSD]2013-05-23 17:49:57.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:57.568 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 866 (0x7f431bfda110) [ CSSD]2013-05-23 17:49:57.568 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda110 (RPC#866) state 6, flags 0x202 [ CSSD]2013-05-23 17:49:57.568 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda110, ret 0, client (nil) [ CSSD]2013-05-23 17:49:57.568 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 866 [ CSSD]2013-05-23 17:49:58.073 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:49:58.073 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:49:58.732 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:49:58.907 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:49:58.908 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:00.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:00.077 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:00.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:02.102 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:02.102 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:02.761 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:02.910 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:02.910 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:50:04.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:04.108 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:04.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:04.355 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015496, gin 1, member 0 [ CSSD]2013-05-23 17:50:04.355 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:50:04.355 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:50:04.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:04.362 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:50:04.362 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:50:04.775 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:06.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:06.140 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:06.781 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:07.591 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:50:07.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 867 (0x7f431bfda190) [ CSSD]2013-05-23 17:50:07.591 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda190 (RPC#867) tag(363002d) sent to node 1 [ CSSD]2013-05-23 17:50:07.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:07.592 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 867 (0x7f431bfda190) [ CSSD]2013-05-23 17:50:07.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda190 (RPC#867) state 6, flags 0x202 [ CSSD]2013-05-23 17:50:07.592 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda190, ret 0, client (nil) [ CSSD]2013-05-23 17:50:07.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 867 [ CSSD]2013-05-23 17:50:07.895 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c151100) allocated [ CSSD]2013-05-23 17:50:07.896 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:50:07.897 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c151100 - 1,2,3 [ CSSD]2013-05-23 17:50:07.897 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14f9e0) proc(0x7f431c151100) pid(11168) version 11:1:1:4 [ CSSD]2013-05-23 17:50:07.913 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:07.913 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:08.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:08.146 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:08.496 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c151100 [ CSSD]2013-05-23 17:50:08.497 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c151100) con(0x7f431c14f9e0) skgpid ospid 11168 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:08.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:09.584 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c142810) allocated [ CSSD]2013-05-23 17:50:09.586 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:50:09.586 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c142810 - 1,2,3 [ CSSD]2013-05-23 17:50:09.586 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c14fab0) proc(0x7f431c142810) pid(11285) version 11:1:1:4 [ CSSD]2013-05-23 17:50:09.591 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c142810), client(1/0x7f431c142770) [ CSSD]2013-05-23 17:50:09.591 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:50:09.591 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c142770 with con 0x7f431c142ae0, requested num -1 [ CSSD]2013-05-23 17:50:09.591 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:09.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 868 (0x7f431bfda210) [ CSSD]2013-05-23 17:50:09.592 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda210 (RPC#868) tag(364002d) sent to node 1 [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0364002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015497, gin 1, member 0 [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1430d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda210 (RPC#868) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda210, ret 0, client 0x7f431c142770 member 0x7f431c1433d0 [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c142770 [ CSSD]2013-05-23 17:50:09.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 868 [ CSSD]2013-05-23 17:50:09.598 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c142770) [ CSSD]2013-05-23 17:50:09.598 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c142770), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:09.598 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1433d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:09.598 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 869 (0x7f431bfda290) [ CSSD]2013-05-23 17:50:09.598 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda290 (RPC#869) tag(365002d) sent to node 1 [ CSSD]2013-05-23 17:50:09.599 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0365002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:09.599 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:09.599 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda290 (RPC#869) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:09.599 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda290, ret 0, client 0x7f431c142770 member 0x7f431c1433d0 [ CSSD]2013-05-23 17:50:09.599 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 869 [ CSSD]2013-05-23 17:50:09.599 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1433d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:09.802 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c142810 [ CSSD]2013-05-23 17:50:09.802 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c142810) con(0x7f431c14fab0) skgpid ospid 11285 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:10.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:10.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:10.839 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:12.182 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:12.182 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:12.845 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:12.916 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:12.916 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:14.188 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:14.188 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:14.856 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:16.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:16.241 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:16.861 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:17.615 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:50:17.615 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 870 (0x7f431bfda310) [ CSSD]2013-05-23 17:50:17.615 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda310 (RPC#870) tag(366002d) sent to node 1 [ CSSD]2013-05-23 17:50:17.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:17.616 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 870 (0x7f431bfda310) [ CSSD]2013-05-23 17:50:17.616 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda310 (RPC#870) state 6, flags 0x202 [ CSSD]2013-05-23 17:50:17.616 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda310, ret 0, client (nil) [ CSSD]2013-05-23 17:50:17.616 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 870 [ CSSD]2013-05-23 17:50:17.919 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:17.919 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:18.247 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:18.247 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:18.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:19.605 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:19.605 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015498, gin 1, member 0 [ CSSD]2013-05-23 17:50:19.605 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:50:19.605 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:50:19.611 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:19.611 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:50:19.611 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:50:20.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:20.257 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:20.624 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c11ab50) [ CSSD]2013-05-23 17:50:20.624 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c11ab50), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:50:20.624 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c11ab50), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:50:20.624 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 29 [ CSSD]2013-05-23 17:50:20.628 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c134690) [ CSSD]2013-05-23 17:50:20.628 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c134690), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:50:20.628 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c134690), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:50:20.628 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 17:50:20.628 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134730 [ CSSD]2013-05-23 17:50:20.628 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134730) con(0x7f431c121820) skgpid 5743 ospid 5743 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:20.880 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:21.922 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:21.922 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:50:22.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:22.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:22.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:24.269 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.269 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:24.418 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c095280) allocated [ CSSD]2013-05-23 17:50:24.419 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:50:24.420 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c095280 - 1,2,3 [ CSSD]2013-05-23 17:50:24.420 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa20) proc(0x7f431c095280) pid(11439) version 11:1:1:4 [ CSSD]2013-05-23 17:50:24.671 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c095280), client(1/0x7f431c14ee00) [ CSSD]2013-05-23 17:50:24.671 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c14ee00) [ CSSD]2013-05-23 17:50:24.672 [1191196992] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_GROUP new client 0x7f431c14ee00 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:50:24.672 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_GROUP [ CSSD]2013-05-23 17:50:24.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 871 (0x7f431bfda390) [ CSSD]2013-05-23 17:50:24.672 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda390 (RPC#871) tag(367002d) sent to node 1 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0367002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock CLSSSCHECK_GROUP, gid 101, gin 1, member 0 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmAddMember: new master 0 for group(CLSSSCHECK_GROUP) [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmAddMember: Adding fencing for member 0, group CLSSSCHECK_GROUP, death 1, SAGE 0 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmAddMember: member (0/0x7f431c134600) added. pbsz(8) prsz(8) flags 0x0 to grock (0x7f431c14f540/CLSSSCHECK_GROUP) [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmQueueGrockEvent: groupName(CLSSSCHECK_GROUP) count(1) master(0) event(1), incarn 1, mbrc 1, to member 0, events 0x0, state 0x0 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmCommonAddMember: global group grock CLSSSCHECK_GROUP member(0/Local) node(2) flags 0x0 0x1000000 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda390 (RPC#871) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda390, ret 0, client 0x7f431c14ee00 member 0x7f431c134600 [ CSSD]2013-05-23 17:50:24.672 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14ee00 [ CSSD]2013-05-23 17:50:24.673 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 871 [ CSSD]2013-05-23 17:50:24.673 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c14ee00) [ CSSD]2013-05-23 17:50:24.673 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c14ee00), grock CLSSSCHECK_GROUP, member 0 [ CSSD]2013-05-23 17:50:24.673 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134600) in global grock CLSSSCHECK_GROUP [ CSSD]2013-05-23 17:50:24.673 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 872 (0x7f431bfda410) [ CSSD]2013-05-23 17:50:24.673 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda410 (RPC#872) tag(368002d) sent to node 1 [ CSSD]2013-05-23 17:50:24.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0368002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.673 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock CLSSSCHECK_GROUP, member 0 [ CSSD]2013-05-23 17:50:24.673 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda410 (RPC#872) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:24.673 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda410, ret 0, client 0x7f431c14ee00 member 0x7f431c134600 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 872 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_GROUP, member number 0 (0x7f431c134600) node number 2 state 0x14 member refcnt 0 grock type 2 [ CSSD]2013-05-23 17:50:24.674 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c095280), client(2/0x7f431c14ee00) [ CSSD]2013-05-23 17:50:24.674 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 2 (0x7f431c14ee00) [ CSSD]2013-05-23 17:50:24.674 [1191196992] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_LOCK new client 0x7f431c14ee00 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:50:24.674 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_LOCK [ CSSD]2013-05-23 17:50:24.674 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 873 (0x7f431bfda490) [ CSSD]2013-05-23 17:50:24.674 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda490 (RPC#873) tag(369002d) sent to node 1 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0369002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock CLSSSCHECK_LOCK, gid 2015499, gin 1, member 0 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c14f540/CLSSSCHECK_LOCK) [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(CLSSSCHECK_LOCK) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock CLSSSCHECK_LOCK member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda490 (RPC#873) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda490, ret 0, client 0x7f431c14ee00 member 0x7f431c135960 [ CSSD]2013-05-23 17:50:24.674 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c14ee00 [ CSSD]2013-05-23 17:50:24.675 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 873 [ CSSD]2013-05-23 17:50:24.675 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 2 (0x7f431c14ee00) [ CSSD]2013-05-23 17:50:24.675 [1191196992] >TRACE: clssgmExitGrock: client 2 (0x7f431c14ee00), grock CLSSSCHECK_LOCK, member 0 [ CSSD]2013-05-23 17:50:24.675 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135960) in global grock CLSSSCHECK_LOCK [ CSSD]2013-05-23 17:50:24.675 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 874 (0x7f431bfda510) [ CSSD]2013-05-23 17:50:24.675 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda510 (RPC#874) tag(36a002d) sent to node 1 [ CSSD]2013-05-23 17:50:24.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(036a002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.675 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock CLSSSCHECK_LOCK, member 0 [ CSSD]2013-05-23 17:50:24.675 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda510 (RPC#874) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:24.675 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda510, ret 0, client 0x7f431c14ee00 member 0x7f431c135960 [ CSSD]2013-05-23 17:50:24.676 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 874 [ CSSD]2013-05-23 17:50:24.676 [1104931136] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_LOCK, member number 0 (0x7f431c135960) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:24.676 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c095280 [ CSSD]2013-05-23 17:50:24.676 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c095280) con(0x7f431c13fa20) skgpid ospid 11439 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:24.833 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:50:24.835 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:50:24.835 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:50:24.835 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c0838d0) pid(11445) version 11:1:1:4 [ CSSD]2013-05-23 17:50:24.852 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c13fa20) [ CSSD]2013-05-23 17:50:24.852 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13fa20) [ CSSD]2013-05-23 17:50:24.852 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13fa20 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:50:24.852 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:24.852 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 875 (0x7f431bfda590) [ CSSD]2013-05-23 17:50:24.852 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda590 (RPC#875) tag(36b002d) sent to node 1 [ CSSD]2013-05-23 17:50:24.852 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(036b002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.852 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015500, gin 1, member 0 [ CSSD]2013-05-23 17:50:24.852 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:50:24.853 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:24.853 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:50:24.853 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda590 (RPC#875) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:24.853 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda590, ret 0, client 0x7f431c13fa20 member 0x7f431c135a50 [ CSSD]2013-05-23 17:50:24.853 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13fa20 [ CSSD]2013-05-23 17:50:24.853 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 875 [ CSSD]2013-05-23 17:50:24.859 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13fa20) [ CSSD]2013-05-23 17:50:24.859 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13fa20), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:24.859 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135a50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:24.859 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 876 (0x7f431bfda610) [ CSSD]2013-05-23 17:50:24.859 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda610 (RPC#876) tag(36c002d) sent to node 1 [ CSSD]2013-05-23 17:50:24.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(036c002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:24.860 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:24.860 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda610 (RPC#876) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:24.860 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda610, ret 0, client 0x7f431c13fa20 member 0x7f431c135a50 [ CSSD]2013-05-23 17:50:24.860 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 876 [ CSSD]2013-05-23 17:50:24.860 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135a50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:24.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:25.062 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:50:25.062 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c11ab50) skgpid ospid 11445 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:26.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:26.281 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:26.925 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:26.925 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:26.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:27.637 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:50:27.637 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 877 (0x7f431bfda690) [ CSSD]2013-05-23 17:50:27.637 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda690 (RPC#877) tag(36d002d) sent to node 1 [ CSSD]2013-05-23 17:50:27.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:27.637 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 877 (0x7f431bfda690) [ CSSD]2013-05-23 17:50:27.637 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda690 (RPC#877) state 6, flags 0x202 [ CSSD]2013-05-23 17:50:27.637 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda690, ret 0, client (nil) [ CSSD]2013-05-23 17:50:27.638 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 877 [ CSSD]2013-05-23 17:50:28.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:28.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:28.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:30.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:30.339 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:30.950 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:31.929 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:31.929 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:32.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:32.344 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:32.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:34.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:34.349 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:34.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:34.853 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015501, gin 1, member 0 [ CSSD]2013-05-23 17:50:34.853 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:50:34.853 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:50:34.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:34.860 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:50:34.860 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:50:34.967 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:36.361 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:36.361 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:36.932 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:36.932 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:36.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:37.674 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:50:37.674 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 878 (0x7f431bfda710) [ CSSD]2013-05-23 17:50:37.674 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda710 (RPC#878) tag(36e002d) sent to node 1 [ CSSD]2013-05-23 17:50:37.675 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:37.675 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 878 (0x7f431bfda710) [ CSSD]2013-05-23 17:50:37.675 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda710 (RPC#878) state 6, flags 0x202 [ CSSD]2013-05-23 17:50:37.675 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda710, ret 0, client (nil) [ CSSD]2013-05-23 17:50:37.675 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 878 [ CSSD]2013-05-23 17:50:38.387 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:38.387 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:38.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:38.762 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015502, gin 1, member 0 [ CSSD]2013-05-23 17:50:38.762 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:50:38.762 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:50:38.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:38.769 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:50:38.769 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:50:39.044 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:40.095 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:50:40.096 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:50:40.097 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:50:40.097 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c121820) proc(0x7f431c0838d0) pid(11592) version 11:1:1:4 [ CSSD]2013-05-23 17:50:40.112 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c135890) [ CSSD]2013-05-23 17:50:40.112 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135890) [ CSSD]2013-05-23 17:50:40.112 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135890 with con 0x7f431c135e60, requested num -1 [ CSSD]2013-05-23 17:50:40.112 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:40.112 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 879 (0x7f431bfda790) [ CSSD]2013-05-23 17:50:40.113 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda790 (RPC#879) tag(36f002d) sent to node 1 [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(036f002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015503, gin 1, member 0 [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda790 (RPC#879) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda790, ret 0, client 0x7f431c135890 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135890 [ CSSD]2013-05-23 17:50:40.113 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 879 [ CSSD]2013-05-23 17:50:40.120 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135890) [ CSSD]2013-05-23 17:50:40.120 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135890), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:40.120 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135ce0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:40.120 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 880 (0x7f431bfda810) [ CSSD]2013-05-23 17:50:40.120 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda810 (RPC#880) tag(370002d) sent to node 1 [ CSSD]2013-05-23 17:50:40.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0370002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:40.120 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:40.120 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda810 (RPC#880) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:40.120 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda810, ret 0, client 0x7f431c135890 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:40.121 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 880 [ CSSD]2013-05-23 17:50:40.121 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135ce0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:40.323 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:50:40.323 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c121820) skgpid ospid 11592 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:40.439 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:40.439 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:40.935 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:40.935 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:50:41.050 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:42.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:42.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:42.645 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:50:42.646 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:50:42.647 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:50:42.647 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa20) proc(0x7f431c0838d0) pid(11654) version 11:1:1:4 [ CSSD]2013-05-23 17:50:42.652 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c11ab50) [ CSSD]2013-05-23 17:50:42.652 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c11ab50) [ CSSD]2013-05-23 17:50:42.652 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c11ab50 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:50:42.652 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:42.652 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 881 (0x7f431bfda890) [ CSSD]2013-05-23 17:50:42.652 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda890 (RPC#881) tag(371002d) sent to node 1 [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0371002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015504, gin 1, member 0 [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda890 (RPC#881) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfda890, ret 0, client 0x7f431c11ab50 member 0x7f431c135a50 [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c11ab50 [ CSSD]2013-05-23 17:50:42.653 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 881 [ CSSD]2013-05-23 17:50:42.660 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c11ab50) [ CSSD]2013-05-23 17:50:42.660 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c11ab50), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:42.660 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135a50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:42.660 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 882 (0x7f431bfda910) [ CSSD]2013-05-23 17:50:42.660 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda910 (RPC#882) tag(372002d) sent to node 1 [ CSSD]2013-05-23 17:50:42.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0372002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:42.660 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:42.660 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda910 (RPC#882) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:42.660 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfda910, ret 0, client 0x7f431c11ab50 member 0x7f431c135a50 [ CSSD]2013-05-23 17:50:42.660 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 882 [ CSSD]2013-05-23 17:50:42.660 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135a50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:42.863 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:50:42.863 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c13fa20) skgpid ospid 11654 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:43.054 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:44.451 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:44.451 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:45.066 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:45.938 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:45.938 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:46.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:46.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:47.073 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:47.699 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:50:47.699 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 883 (0x7f431bfda990) [ CSSD]2013-05-23 17:50:47.700 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfda990 (RPC#883) tag(373002d) sent to node 1 [ CSSD]2013-05-23 17:50:47.700 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:47.700 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 883 (0x7f431bfda990) [ CSSD]2013-05-23 17:50:47.700 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfda990 (RPC#883) state 6, flags 0x202 [ CSSD]2013-05-23 17:50:47.700 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfda990, ret 0, client (nil) [ CSSD]2013-05-23 17:50:47.700 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 883 [ CSSD]2013-05-23 17:50:48.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:48.468 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:49.084 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:50.101 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:50.101 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015505, gin 1, member 0 [ CSSD]2013-05-23 17:50:50.102 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:50:50.102 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:50:50.108 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:50.108 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:50:50.108 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:50:50.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:50.479 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:50.941 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:50.942 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:51.128 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:51.955 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:50:51.955 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:50:51.956 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:50:51.956 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c121820) proc(0x7f431c0838d0) pid(11743) version 11:1:1:4 [ CSSD]2013-05-23 17:50:52.032 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c135e70) [ CSSD]2013-05-23 17:50:52.032 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135e70) [ CSSD]2013-05-23 17:50:52.032 [1191196992] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_GROUP new client 0x7f431c135e70 with con 0x7f431c134600, requested num -1 [ CSSD]2013-05-23 17:50:52.032 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_GROUP [ CSSD]2013-05-23 17:50:52.032 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 884 (0x7f431bfdaa10) [ CSSD]2013-05-23 17:50:52.032 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaa10 (RPC#884) tag(374002d) sent to node 1 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0374002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock CLSSSCHECK_GROUP, gid 102, gin 1, member 0 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmAddMember: new master 0 for group(CLSSSCHECK_GROUP) [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmAddMember: Adding fencing for member 0, group CLSSSCHECK_GROUP, death 1, SAGE 0 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmAddMember: member (0/0x7f431c135ce0) added. pbsz(8) prsz(8) flags 0x0 to grock (0x7f431c14f540/CLSSSCHECK_GROUP) [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmQueueGrockEvent: groupName(CLSSSCHECK_GROUP) count(1) master(0) event(1), incarn 1, mbrc 1, to member 0, events 0x0, state 0x0 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmCommonAddMember: global group grock CLSSSCHECK_GROUP member(0/Local) node(2) flags 0x0 0x1000000 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaa10 (RPC#884) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdaa10, ret 0, client 0x7f431c135e70 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135e70 [ CSSD]2013-05-23 17:50:52.033 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 884 [ CSSD]2013-05-23 17:50:52.033 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135e70) [ CSSD]2013-05-23 17:50:52.033 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135e70), grock CLSSSCHECK_GROUP, member 0 [ CSSD]2013-05-23 17:50:52.033 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135ce0) in global grock CLSSSCHECK_GROUP [ CSSD]2013-05-23 17:50:52.033 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 885 (0x7f431bfdaa90) [ CSSD]2013-05-23 17:50:52.034 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaa90 (RPC#885) tag(375002d) sent to node 1 [ CSSD]2013-05-23 17:50:52.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0375002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:52.034 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock CLSSSCHECK_GROUP, member 0 [ CSSD]2013-05-23 17:50:52.034 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaa90 (RPC#885) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:52.034 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdaa90, ret 0, client 0x7f431c135e70 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:52.034 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 885 [ CSSD]2013-05-23 17:50:52.034 [1104931136] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_GROUP, member number 0 (0x7f431c135ce0) node number 2 state 0x14 member refcnt 0 grock type 2 [ CSSD]2013-05-23 17:50:52.034 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(2/0x7f431c135e60) [ CSSD]2013-05-23 17:50:52.034 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 2 (0x7f431c135e60) [ CSSD]2013-05-23 17:50:52.034 [1191196992] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_LOCK new client 0x7f431c135e60 with con 0x7f431c13fa20, requested num -1 [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_LOCK [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 886 (0x7f431bfdab10) [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdab10 (RPC#886) tag(376002d) sent to node 1 [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0376002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock CLSSSCHECK_LOCK, gid 2015506, gin 1, member 0 [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c14f540/CLSSSCHECK_LOCK) [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(CLSSSCHECK_LOCK) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock CLSSSCHECK_LOCK member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdab10 (RPC#886) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdab10, ret 0, client 0x7f431c135e60 member 0x7f431c135960 [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135e60 [ CSSD]2013-05-23 17:50:52.035 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 886 [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 2 (0x7f431c135e60) [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmExitGrock: client 2 (0x7f431c135e60), grock CLSSSCHECK_LOCK, member 0 [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135960) in global grock CLSSSCHECK_LOCK [ CSSD]2013-05-23 17:50:52.035 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 887 (0x7f431bfdab90) [ CSSD]2013-05-23 17:50:52.036 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdab90 (RPC#887) tag(377002d) sent to node 1 [ CSSD]2013-05-23 17:50:52.036 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0377002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:52.036 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock CLSSSCHECK_LOCK, member 0 [ CSSD]2013-05-23 17:50:52.036 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdab90 (RPC#887) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:52.036 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdab90, ret 0, client 0x7f431c135e60 member 0x7f431c135960 [ CSSD]2013-05-23 17:50:52.036 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 887 [ CSSD]2013-05-23 17:50:52.036 [1104931136] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_LOCK, member number 0 (0x7f431c135960) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:52.036 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:50:52.036 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c121820) skgpid ospid 11743 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:52.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:52.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:53.134 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:54.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:54.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:55.167 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:55.365 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:50:55.366 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:50:55.366 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:50:55.366 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa20) proc(0x7f431c0838d0) pid(11778) version 11:1:1:4 [ CSSD]2013-05-23 17:50:55.373 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c135890) [ CSSD]2013-05-23 17:50:55.373 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135890) [ CSSD]2013-05-23 17:50:55.373 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135890 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:50:55.373 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:55.373 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 888 (0x7f431bfdac10) [ CSSD]2013-05-23 17:50:55.373 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdac10 (RPC#888) tag(378002d) sent to node 1 [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0378002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015507, gin 1, member 0 [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdac10 (RPC#888) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdac10, ret 0, client 0x7f431c135890 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135890 [ CSSD]2013-05-23 17:50:55.374 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 888 [ CSSD]2013-05-23 17:50:55.381 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135890) [ CSSD]2013-05-23 17:50:55.381 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135890), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:55.381 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135ce0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:55.381 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 889 (0x7f431bfdac90) [ CSSD]2013-05-23 17:50:55.381 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdac90 (RPC#889) tag(379002d) sent to node 1 [ CSSD]2013-05-23 17:50:55.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0379002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:55.381 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:55.381 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdac90 (RPC#889) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:55.381 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdac90, ret 0, client 0x7f431c135890 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:55.382 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 889 [ CSSD]2013-05-23 17:50:55.382 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135ce0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:55.584 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:50:55.584 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c13fa20) skgpid ospid 11778 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:50:55.945 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:50:55.945 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:50:56.542 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:56.542 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:57.173 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:57.722 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:50:57.722 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 890 (0x7f431bfdad10) [ CSSD]2013-05-23 17:50:57.723 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdad10 (RPC#890) tag(37a002d) sent to node 1 [ CSSD]2013-05-23 17:50:57.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:57.723 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 890 (0x7f431bfdad10) [ CSSD]2013-05-23 17:50:57.723 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdad10 (RPC#890) state 6, flags 0x202 [ CSSD]2013-05-23 17:50:57.723 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdad10, ret 0, client (nil) [ CSSD]2013-05-23 17:50:57.723 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 890 [ CSSD]2013-05-23 17:50:58.567 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:50:58.567 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:50:59.185 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:50:59.755 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:50:59.773 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:50:59.774 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:50:59.774 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c0838d0) pid(11846) version 11:1:1:4 [ CSSD]2013-05-23 17:50:59.780 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c135890) [ CSSD]2013-05-23 17:50:59.780 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135890) [ CSSD]2013-05-23 17:50:59.780 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135890 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:50:59.780 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:59.780 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 891 (0x7f431bfdad90) [ CSSD]2013-05-23 17:50:59.780 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdad90 (RPC#891) tag(37b002d) sent to node 1 [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(037b002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015508, gin 1, member 0 [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdad90 (RPC#891) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdad90, ret 0, client 0x7f431c135890 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135890 [ CSSD]2013-05-23 17:50:59.781 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 891 [ CSSD]2013-05-23 17:50:59.787 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135890) [ CSSD]2013-05-23 17:50:59.787 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135890), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:59.787 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135ce0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:50:59.787 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 892 (0x7f431bfdae10) [ CSSD]2013-05-23 17:50:59.787 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdae10 (RPC#892) tag(37c002d) sent to node 1 [ CSSD]2013-05-23 17:50:59.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(037c002d) incarnation(174631051) [ CSSD]2013-05-23 17:50:59.787 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:50:59.787 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdae10 (RPC#892) state 6, flags 0x100 [ CSSD]2013-05-23 17:50:59.787 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdae10, ret 0, client 0x7f431c135890 member 0x7f431c135ce0 [ CSSD]2013-05-23 17:50:59.788 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 892 [ CSSD]2013-05-23 17:50:59.788 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135ce0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:50:59.990 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:50:59.990 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c11ab50) skgpid ospid 11846 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:00.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:00.578 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:00.948 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:00.948 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:01.191 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:02.584 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:02.584 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:03.197 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:04.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:04.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:05.248 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:05.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:05.363 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015509, gin 1, member 0 [ CSSD]2013-05-23 17:51:05.363 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:51:05.363 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:51:05.370 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:05.370 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:51:05.370 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:51:05.951 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:05.951 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:06.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:06.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:07.273 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:07.758 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:51:07.758 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 893 (0x7f431bfdae90) [ CSSD]2013-05-23 17:51:07.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdae90 (RPC#893) tag(37d002d) sent to node 1 [ CSSD]2013-05-23 17:51:07.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:07.759 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 893 (0x7f431bfdae90) [ CSSD]2013-05-23 17:51:07.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdae90 (RPC#893) state 6, flags 0x202 [ CSSD]2013-05-23 17:51:07.759 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdae90, ret 0, client (nil) [ CSSD]2013-05-23 17:51:07.759 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 893 [ CSSD]2013-05-23 17:51:08.649 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:08.649 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:09.304 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:10.617 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:51:10.618 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:51:10.618 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:51:10.618 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c0838d0) pid(11929) version 11:1:1:4 [ CSSD]2013-05-23 17:51:10.624 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c135e60) [ CSSD]2013-05-23 17:51:10.624 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135e60) [ CSSD]2013-05-23 17:51:10.624 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135e60 with con 0x7f431c121820, requested num -1 [ CSSD]2013-05-23 17:51:10.624 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:10.624 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 894 (0x7f431bfdaf10) [ CSSD]2013-05-23 17:51:10.624 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaf10 (RPC#894) tag(37e002d) sent to node 1 [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(037e002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015510, gin 1, member 0 [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c14f540/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaf10 (RPC#894) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdaf10, ret 0, client 0x7f431c135e60 member 0x7f431c135960 [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135e60 [ CSSD]2013-05-23 17:51:10.625 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 894 [ CSSD]2013-05-23 17:51:10.632 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135e60) [ CSSD]2013-05-23 17:51:10.632 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135e60), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:10.632 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135960) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:10.632 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 895 (0x7f431bfdaf90) [ CSSD]2013-05-23 17:51:10.632 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdaf90 (RPC#895) tag(37f002d) sent to node 1 [ CSSD]2013-05-23 17:51:10.633 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(037f002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:10.633 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:10.633 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdaf90 (RPC#895) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:10.633 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdaf90, ret 0, client 0x7f431c135e60 member 0x7f431c135960 [ CSSD]2013-05-23 17:51:10.633 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 895 [ CSSD]2013-05-23 17:51:10.633 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135960) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:51:10.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:10.673 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:10.835 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:51:10.835 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c11ab50) skgpid ospid 11929 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:10.953 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:10.954 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:11.310 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:12.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:12.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:13.343 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:14.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:14.671 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:14.760 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0838d0) allocated [ CSSD]2013-05-23 17:51:14.761 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:51:14.762 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0838d0 - 1,2,3 [ CSSD]2013-05-23 17:51:14.762 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13fa20) proc(0x7f431c0838d0) pid(11414/11414) version 11:1:1:4 [ CSSD]2013-05-23 17:51:14.767 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(1/0x7f431c11ab50) [ CSSD]2013-05-23 17:51:14.767 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c11ab50) [ CSSD]2013-05-23 17:51:14.768 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:51:14.768 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 17:51:15.348 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:15.956 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:15.956 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:16.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:16.683 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:17.353 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:17.787 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:51:17.787 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 896 (0x7f431bfdb010) [ CSSD]2013-05-23 17:51:17.787 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb010 (RPC#896) tag(380002d) sent to node 1 [ CSSD]2013-05-23 17:51:17.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:17.788 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 896 (0x7f431bfdb010) [ CSSD]2013-05-23 17:51:17.788 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb010 (RPC#896) state 6, flags 0x202 [ CSSD]2013-05-23 17:51:17.788 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb010, ret 0, client (nil) [ CSSD]2013-05-23 17:51:17.788 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 896 [ CSSD]2013-05-23 17:51:18.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:18.689 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:19.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:20.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:20.614 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015511, gin 1, member 0 [ CSSD]2013-05-23 17:51:20.614 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:51:20.614 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:51:20.621 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:20.621 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:51:20.621 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2319290) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:51:20.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:20.711 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:20.959 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:20.959 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:21.371 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:22.679 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1510e0) allocated [ CSSD]2013-05-23 17:51:22.680 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:51:22.680 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1510e0 - 1,2,3 [ CSSD]2013-05-23 17:51:22.680 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095340) proc(0x7f431c1510e0) pid(12058) version 11:1:1:4 [ CSSD]2013-05-23 17:51:22.687 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c1510e0), client(1/0x7f431c151040) [ CSSD]2013-05-23 17:51:22.687 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c151040) [ CSSD]2013-05-23 17:51:22.687 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c151040 with con 0x7f431c142750, requested num -1 [ CSSD]2013-05-23 17:51:22.687 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:51:22.687 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 897 (0x7f431bfdb090) [ CSSD]2013-05-23 17:51:22.687 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb090 (RPC#897) tag(381002d) sent to node 1 [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0381002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015512, gin 1, member 0 [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c142d40/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb090 (RPC#897) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb090, ret 0, client 0x7f431c151040 member 0x7f431c143040 [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c151040 [ CSSD]2013-05-23 17:51:22.688 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 897 [ CSSD]2013-05-23 17:51:22.694 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c151040) [ CSSD]2013-05-23 17:51:22.694 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c151040), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:51:22.694 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c143040) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:51:22.694 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 898 (0x7f431bfdb110) [ CSSD]2013-05-23 17:51:22.694 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb110 (RPC#898) tag(382002d) sent to node 1 [ CSSD]2013-05-23 17:51:22.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0382002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:22.695 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:51:22.695 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb110 (RPC#898) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:22.695 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb110, ret 0, client 0x7f431c151040 member 0x7f431c143040 [ CSSD]2013-05-23 17:51:22.695 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 898 [ CSSD]2013-05-23 17:51:22.695 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c143040) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:51:22.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:22.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:22.814 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c144ec0) allocated [ CSSD]2013-05-23 17:51:22.815 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:51:22.815 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c144ec0 - 1,2,3 [ CSSD]2013-05-23 17:51:22.815 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135aa0) proc(0x7f431c144ec0) pid(12063) version 11:1:1:4 [ CSSD]2013-05-23 17:51:22.841 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:51:22.842 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:51:22.844 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c144ec0 [ CSSD]2013-05-23 17:51:22.845 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c144ec0) con(0x7f431c135aa0) skgpid ospid 12063 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:22.998 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1510e0 [ CSSD]2013-05-23 17:51:22.998 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1510e0) con(0x7f431c095340) skgpid ospid 12058 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:23.444 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:24.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:24.723 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:24.962 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:24.962 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:51:25.221 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c1306d0) [ CSSD]2013-05-23 17:51:25.221 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c1306d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:51:25.221 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c1306d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:51:25.221 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 28 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c135350) [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c135350), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c135350), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 27 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c132e60 [ CSSD]2013-05-23 17:51:25.225 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c132e60) con(0x7f431c0e5900) skgpid 6442 ospid 6442 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:25.230 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c13f730) [ CSSD]2013-05-23 17:51:25.230 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c13f730), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:51:25.230 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c13f730), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:51:25.230 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 17:51:25.234 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:51:25.234 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:51:25.234 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:51:25.234 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 17:51:25.234 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:51:25.235 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12f0c0) skgpid 6446 ospid 6446 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:25.238 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23193a0) [ CSSD]2013-05-23 17:51:25.238 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23193a0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:51:25.238 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23193a0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:51:25.238 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:51:25.238 [1191196992] >TRACE: clssgmDeadProc: proc 0x2318e90 [ CSSD]2013-05-23 17:51:25.238 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2318e90) con(0x22ec9a0) skgpid 4296 ospid 4296 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:25.239 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c130630) [ CSSD]2013-05-23 17:51:25.239 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c130630), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:51:25.239 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c130630), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:51:25.239 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:51:25.244 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:51:25.244 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c108e70), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:51:25.244 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c108e70), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:51:25.244 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:51:25.245 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13ea10 [ CSSD]2013-05-23 17:51:25.245 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13ea10) con(0x7f431c0e6720) skgpid 6444 ospid 6444 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:25.468 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:25.875 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:51:25.876 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:51:25.877 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:51:25.877 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c13f020) proc(0x7f431c1355f0) pid(12090) version 11:1:1:4 [ CSSD]2013-05-23 17:51:25.883 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 17:51:25.883 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:51:25.883 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:51:25.883 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:25.883 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 899 (0x7f431bfdb190) [ CSSD]2013-05-23 17:51:25.883 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb190 (RPC#899) tag(383002d) sent to node 1 [ CSSD]2013-05-23 17:51:25.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0383002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:25.883 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015513, gin 1, member 0 [ CSSD]2013-05-23 17:51:25.883 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:51:25.884 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:51:25.884 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:51:25.884 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb190 (RPC#899) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:25.884 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb190, ret 0, client 0x7f431c12e6d0 member 0x7f431c081ae0 [ CSSD]2013-05-23 17:51:25.884 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 17:51:25.884 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 899 [ CSSD]2013-05-23 17:51:25.889 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:51:25.889 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:25.889 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c081ae0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:25.889 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 900 (0x7f431bfdb210) [ CSSD]2013-05-23 17:51:25.889 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb210 (RPC#900) tag(384002d) sent to node 1 [ CSSD]2013-05-23 17:51:25.890 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0384002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:25.890 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:25.890 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb210 (RPC#900) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:25.890 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb210, ret 0, client 0x7f431c12e6d0 member 0x7f431c081ae0 [ CSSD]2013-05-23 17:51:25.890 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 900 [ CSSD]2013-05-23 17:51:25.890 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c081ae0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:51:26.093 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:51:26.093 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c13f020) skgpid ospid 12090 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:26.754 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:26.754 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:27.473 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:27.822 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:51:27.822 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 901 (0x7f431bfdb290) [ CSSD]2013-05-23 17:51:27.822 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb290 (RPC#901) tag(385002d) sent to node 1 [ CSSD]2013-05-23 17:51:27.823 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:27.823 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 901 (0x7f431bfdb290) [ CSSD]2013-05-23 17:51:27.823 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb290 (RPC#901) state 6, flags 0x202 [ CSSD]2013-05-23 17:51:27.823 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb290, ret 0, client (nil) [ CSSD]2013-05-23 17:51:27.823 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 901 [ CSSD]2013-05-23 17:51:28.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:28.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:29.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:29.965 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:29.965 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:30.852 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:30.852 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:31.492 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:32.077 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0838d0), client(2/0x7f431c12e6d0) [ CSSD]2013-05-23 17:51:32.077 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:51:32.077 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:51:32.077 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:51:32.858 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:32.858 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:33.544 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:34.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:34.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:34.968 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:34.968 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:35.570 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:35.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:35.864 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015514, gin 1, member 0 [ CSSD]2013-05-23 17:51:35.864 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:51:35.864 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:51:35.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:35.871 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:51:35.871 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:51:36.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:36.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:37.576 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:37.850 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:51:37.850 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 902 (0x7f431bfdb310) [ CSSD]2013-05-23 17:51:37.850 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb310 (RPC#902) tag(386002d) sent to node 1 [ CSSD]2013-05-23 17:51:37.851 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:37.851 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 902 (0x7f431bfdb310) [ CSSD]2013-05-23 17:51:37.851 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb310 (RPC#902) state 6, flags 0x202 [ CSSD]2013-05-23 17:51:37.851 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb310, ret 0, client (nil) [ CSSD]2013-05-23 17:51:37.851 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 902 [ CSSD]2013-05-23 17:51:38.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:38.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:39.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:39.025 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015515, gin 1, member 0 [ CSSD]2013-05-23 17:51:39.025 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:51:39.025 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:51:39.031 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:39.031 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:51:39.031 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:51:39.588 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:39.970 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:39.971 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:40.952 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:40.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:41.132 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:51:41.133 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:51:41.133 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:51:41.133 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135f10) proc(0x7f431c1355f0) pid(12266) version 11:1:1:4 [ CSSD]2013-05-23 17:51:41.139 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c12a550) [ CSSD]2013-05-23 17:51:41.139 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12a550) [ CSSD]2013-05-23 17:51:41.139 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12a550 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:51:41.139 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:41.139 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 903 (0x7f431bfdb390) [ CSSD]2013-05-23 17:51:41.139 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb390 (RPC#903) tag(387002d) sent to node 1 [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0387002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015516, gin 1, member 0 [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb390 (RPC#903) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb390, ret 0, client 0x7f431c12a550 member 0x7f431c108da0 [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12a550 [ CSSD]2013-05-23 17:51:41.140 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 903 [ CSSD]2013-05-23 17:51:41.146 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12a550) [ CSSD]2013-05-23 17:51:41.146 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12a550), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:41.146 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:41.146 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 904 (0x7f431bfdb410) [ CSSD]2013-05-23 17:51:41.146 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb410 (RPC#904) tag(388002d) sent to node 1 [ CSSD]2013-05-23 17:51:41.146 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0388002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:41.146 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:41.146 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb410 (RPC#904) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:41.146 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb410, ret 0, client 0x7f431c12a550 member 0x7f431c108da0 [ CSSD]2013-05-23 17:51:41.147 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 904 [ CSSD]2013-05-23 17:51:41.147 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:51:41.349 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:51:41.349 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c135f10) skgpid ospid 12266 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:41.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:42.913 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:51:42.914 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:51:42.914 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:51:42.914 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1355f0) pid(12323) version 11:1:1:4 [ CSSD]2013-05-23 17:51:42.920 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c12a550) [ CSSD]2013-05-23 17:51:42.920 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12a550) [ CSSD]2013-05-23 17:51:42.920 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12a550 with con 0x7f431c135f10, requested num -1 [ CSSD]2013-05-23 17:51:42.920 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:42.920 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 905 (0x7f431bfdb490) [ CSSD]2013-05-23 17:51:42.920 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb490 (RPC#905) tag(389002d) sent to node 1 [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0389002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015517, gin 1, member 0 [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb490 (RPC#905) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb490, ret 0, client 0x7f431c12a550 member 0x7f431c10a490 [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12a550 [ CSSD]2013-05-23 17:51:42.921 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 905 [ CSSD]2013-05-23 17:51:42.928 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12a550) [ CSSD]2013-05-23 17:51:42.928 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12a550), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:42.928 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:42.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 906 (0x7f431bfdb510) [ CSSD]2013-05-23 17:51:42.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb510 (RPC#906) tag(38a002d) sent to node 1 [ CSSD]2013-05-23 17:51:42.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(038a002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:42.928 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:42.928 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb510 (RPC#906) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:42.928 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb510, ret 0, client 0x7f431c12a550 member 0x7f431c10a490 [ CSSD]2013-05-23 17:51:42.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 906 [ CSSD]2013-05-23 17:51:42.929 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:51:42.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:42.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:43.131 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:51:43.131 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12f0c0) skgpid ospid 12323 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:43.594 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:44.969 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:44.969 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:44.974 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:44.974 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:45.600 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:46.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:46.995 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:47.626 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:47.878 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:51:47.878 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 907 (0x7f431bfdb590) [ CSSD]2013-05-23 17:51:47.878 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb590 (RPC#907) tag(38b002d) sent to node 1 [ CSSD]2013-05-23 17:51:47.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:47.879 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 907 (0x7f431bfdb590) [ CSSD]2013-05-23 17:51:47.879 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb590 (RPC#907) state 6, flags 0x202 [ CSSD]2013-05-23 17:51:47.879 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb590, ret 0, client (nil) [ CSSD]2013-05-23 17:51:47.879 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 907 [ CSSD]2013-05-23 17:51:48.976 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:48.976 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:51:49.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:49.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:49.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:51.052 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:51.053 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:51.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:51.122 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015518, gin 1, member 0 [ CSSD]2013-05-23 17:51:51.122 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:51:51.122 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:51:51.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:51.129 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:51:51.129 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:51:51.683 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:53.058 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:53.058 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:53.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:53.979 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:53.979 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:55.069 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:55.069 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:55.700 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:56.382 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:51:56.383 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:51:56.384 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:51:56.384 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1355f0) pid(12417) version 11:1:1:4 [ CSSD]2013-05-23 17:51:56.389 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c12a550) [ CSSD]2013-05-23 17:51:56.389 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12a550) [ CSSD]2013-05-23 17:51:56.389 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12a550 with con 0x7f431c13f020, requested num -1 [ CSSD]2013-05-23 17:51:56.389 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:56.389 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 908 (0x7f431bfdb610) [ CSSD]2013-05-23 17:51:56.389 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb610 (RPC#908) tag(38c002d) sent to node 1 [ CSSD]2013-05-23 17:51:56.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(038c002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:56.389 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015519, gin 1, member 0 [ CSSD]2013-05-23 17:51:56.389 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:51:56.390 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:51:56.390 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:51:56.390 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb610 (RPC#908) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:56.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb610, ret 0, client 0x7f431c12a550 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:51:56.390 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12a550 [ CSSD]2013-05-23 17:51:56.390 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 908 [ CSSD]2013-05-23 17:51:56.396 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12a550) [ CSSD]2013-05-23 17:51:56.396 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12a550), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:56.396 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:51:56.396 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 909 (0x7f431bfdb690) [ CSSD]2013-05-23 17:51:56.396 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb690 (RPC#909) tag(38d002d) sent to node 1 [ CSSD]2013-05-23 17:51:56.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(038d002d) incarnation(174631051) [ CSSD]2013-05-23 17:51:56.396 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:51:56.396 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb690 (RPC#909) state 6, flags 0x100 [ CSSD]2013-05-23 17:51:56.396 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb690, ret 0, client 0x7f431c12a550 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:51:56.397 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 909 [ CSSD]2013-05-23 17:51:56.397 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:51:56.599 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:51:56.599 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c0e6720) skgpid ospid 12417 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:51:57.075 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:57.075 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:57.725 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:51:57.903 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:51:57.904 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 910 (0x7f431bfdb710) [ CSSD]2013-05-23 17:51:57.904 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb710 (RPC#910) tag(38e002d) sent to node 1 [ CSSD]2013-05-23 17:51:57.905 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:57.905 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 910 (0x7f431bfdb710) [ CSSD]2013-05-23 17:51:57.905 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb710 (RPC#910) state 6, flags 0x202 [ CSSD]2013-05-23 17:51:57.905 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb710, ret 0, client (nil) [ CSSD]2013-05-23 17:51:57.905 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 910 [ CSSD]2013-05-23 17:51:58.982 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:51:58.982 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:51:59.081 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:51:59.081 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:51:59.778 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:01.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:01.093 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:01.784 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:03.099 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:03.099 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:03.796 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:03.985 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:03.985 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:05.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:05.151 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:05.802 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:06.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:06.384 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015520, gin 1, member 0 [ CSSD]2013-05-23 17:52:06.384 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:52:06.384 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:52:06.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:06.392 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:52:06.392 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:52:07.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:07.156 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:07.807 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:07.931 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:52:07.931 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 911 (0x7f431bfdb790) [ CSSD]2013-05-23 17:52:07.931 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb790 (RPC#911) tag(38f002d) sent to node 1 [ CSSD]2013-05-23 17:52:07.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:07.932 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 911 (0x7f431bfdb790) [ CSSD]2013-05-23 17:52:07.932 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb790 (RPC#911) state 6, flags 0x202 [ CSSD]2013-05-23 17:52:07.932 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdb790, ret 0, client (nil) [ CSSD]2013-05-23 17:52:07.932 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 911 [ CSSD]2013-05-23 17:52:08.988 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:08.988 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:09.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:09.163 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:09.801 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:52:09.802 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:52:09.802 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:52:09.802 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c1355f0) pid(12552) version 11:1:1:4 [ CSSD]2013-05-23 17:52:09.808 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c1355f0), client(1/0x7f431c13f020) [ CSSD]2013-05-23 17:52:09.808 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c13f020) [ CSSD]2013-05-23 17:52:09.808 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c13f020 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 17:52:09.808 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:09.808 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 912 (0x7f431bfdb810) [ CSSD]2013-05-23 17:52:09.809 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb810 (RPC#912) tag(390002d) sent to node 1 [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0390002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015521, gin 1, member 0 [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0951b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb810 (RPC#912) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb810, ret 0, client 0x7f431c13f020 member 0x7f431c10a490 [ CSSD]2013-05-23 17:52:09.809 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f020 [ CSSD]2013-05-23 17:52:09.810 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 912 [ CSSD]2013-05-23 17:52:09.816 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c13f020) [ CSSD]2013-05-23 17:52:09.816 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c13f020), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:09.816 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c10a490) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:09.816 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 913 (0x7f431bfdb890) [ CSSD]2013-05-23 17:52:09.816 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb890 (RPC#913) tag(391002d) sent to node 1 [ CSSD]2013-05-23 17:52:09.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0391002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:09.816 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:09.816 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb890 (RPC#913) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:09.816 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb890, ret 0, client 0x7f431c13f020 member 0x7f431c10a490 [ CSSD]2013-05-23 17:52:09.817 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 913 [ CSSD]2013-05-23 17:52:09.817 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c10a490) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:09.878 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:10.019 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:52:10.019 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12a550) skgpid ospid 12552 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:11.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:11.174 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:11.631 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:52:11.632 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:52:11.633 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:52:11.633 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22fdb60) pid(12576) version 11:1:1:4 [ CSSD]2013-05-23 17:52:11.639 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x22d7da0) [ CSSD]2013-05-23 17:52:11.639 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 17:52:11.639 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7da0 with con 0x22e2f70, requested num -1 [ CSSD]2013-05-23 17:52:11.639 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:11.639 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 914 (0x7f431bfdb910) [ CSSD]2013-05-23 17:52:11.639 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb910 (RPC#914) tag(392002d) sent to node 1 [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0392002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015522, gin 1, member 0 [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb910 (RPC#914) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdb910, ret 0, client 0x22d7da0 member 0x22fc9c0 [ CSSD]2013-05-23 17:52:11.639 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7da0 [ CSSD]2013-05-23 17:52:11.640 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 914 [ CSSD]2013-05-23 17:52:11.645 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 17:52:11.646 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7da0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:11.646 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc9c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:11.646 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 915 (0x7f431bfdb990) [ CSSD]2013-05-23 17:52:11.646 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdb990 (RPC#915) tag(393002d) sent to node 1 [ CSSD]2013-05-23 17:52:11.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0393002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:11.646 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:11.646 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdb990 (RPC#915) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:11.646 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdb990, ret 0, client 0x22d7da0 member 0x22fc9c0 [ CSSD]2013-05-23 17:52:11.646 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 915 [ CSSD]2013-05-23 17:52:11.646 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc9c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:11.849 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:52:11.849 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x2312880) skgpid ospid 12576 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:11.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:12.990 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:12.990 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:52:13.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:13.219 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:13.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:15.251 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:15.251 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:15.961 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:17.257 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:17.257 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:17.955 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:52:17.955 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 916 (0x7f431bfdba10) [ CSSD]2013-05-23 17:52:17.955 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdba10 (RPC#916) tag(394002d) sent to node 1 [ CSSD]2013-05-23 17:52:17.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:17.956 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 916 (0x7f431bfdba10) [ CSSD]2013-05-23 17:52:17.956 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdba10 (RPC#916) state 6, flags 0x202 [ CSSD]2013-05-23 17:52:17.956 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdba10, ret 0, client (nil) [ CSSD]2013-05-23 17:52:17.956 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 916 [ CSSD]2013-05-23 17:52:17.993 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:17.993 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:18.005 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:19.283 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:19.283 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:20.080 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:21.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:21.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:21.482 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 17:52:21.483 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:52:21.484 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 17:52:21.484 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22dacd0) pid(12898) version 11:1:1:4 [ CSSD]2013-05-23 17:52:21.604 [1191196992] >TRACE: clssgmAllocProc: (0x2319290) allocated [ CSSD]2013-05-23 17:52:21.605 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:52:21.606 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2319290 - 1,2,3 [ CSSD]2013-05-23 17:52:21.606 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7af0) proc(0x2319290) pid(12908) version 11:1:1:4 [ CSSD]2013-05-23 17:52:21.621 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:52:21.623 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:52:21.625 [1191196992] >TRACE: clssgmDeadProc: proc 0x2319290 [ CSSD]2013-05-23 17:52:21.626 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2319290) con(0x22d7af0) skgpid ospid 12908 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:21.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:21.645 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015523, gin 1, member 0 [ CSSD]2013-05-23 17:52:21.645 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:52:21.645 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:52:21.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:21.652 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:52:21.652 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22d7a00) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:52:21.705 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(85/0x22d7da0) [ CSSD]2013-05-23 17:52:21.705 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 85 (0x22d7da0) [ CSSD]2013-05-23 17:52:21.705 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x22d7da0 with con 0x231d790, requested num -1 [ CSSD]2013-05-23 17:52:21.705 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:52:21.705 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 917 (0x7f431bfdba90) [ CSSD]2013-05-23 17:52:21.705 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdba90 (RPC#917) tag(395002d) sent to node 1 [ CSSD]2013-05-23 17:52:21.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0395002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:21.705 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015524, gin 1, member 0 [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x2302b40/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdba90 (RPC#917) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdba90, ret 0, client 0x22d7da0 member 0x22d7ad0 [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7da0 [ CSSD]2013-05-23 17:52:21.706 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 917 [ CSSD]2013-05-23 17:52:21.712 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 85 (0x22d7da0) [ CSSD]2013-05-23 17:52:21.712 [1191196992] >TRACE: clssgmExitGrock: client 85 (0x22d7da0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:52:21.712 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7ad0) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:52:21.712 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 918 (0x7f431bfdbb10) [ CSSD]2013-05-23 17:52:21.712 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbb10 (RPC#918) tag(396002d) sent to node 1 [ CSSD]2013-05-23 17:52:21.712 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0396002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:21.712 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:52:21.712 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbb10 (RPC#918) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:21.712 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbb10, ret 0, client 0x22d7da0 member 0x22d7ad0 [ CSSD]2013-05-23 17:52:21.712 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 918 [ CSSD]2013-05-23 17:52:21.712 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x22d7ad0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:21.714 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 17:52:21.714 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2312880) skgpid ospid 12898 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:22.086 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:22.995 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:22.995 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:23.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:23.300 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:24.096 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:25.352 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:25.352 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:26.103 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:26.885 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 17:52:26.886 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:52:26.886 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 17:52:26.886 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7d90) proc(0x22fc9c0) pid(12950) version 11:1:1:4 [ CSSD]2013-05-23 17:52:26.892 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fc9c0), client(1/0x2312950) [ CSSD]2013-05-23 17:52:26.892 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:52:26.892 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 17:52:26.892 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:26.892 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 919 (0x7f431bfdbb90) [ CSSD]2013-05-23 17:52:26.893 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbb90 (RPC#919) tag(397002d) sent to node 1 [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0397002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015525, gin 1, member 0 [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbb90 (RPC#919) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdbb90, ret 0, client 0x2312950 member 0x2319290 [ CSSD]2013-05-23 17:52:26.893 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 17:52:26.894 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 919 [ CSSD]2013-05-23 17:52:26.900 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:52:26.900 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:26.900 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2319290) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:26.900 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 920 (0x7f431bfdbc10) [ CSSD]2013-05-23 17:52:26.900 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbc10 (RPC#920) tag(398002d) sent to node 1 [ CSSD]2013-05-23 17:52:26.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0398002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:26.900 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:26.900 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbc10 (RPC#920) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:26.900 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbc10, ret 0, client 0x2312950 member 0x2319290 [ CSSD]2013-05-23 17:52:26.901 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 920 [ CSSD]2013-05-23 17:52:26.901 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2319290) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:27.103 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 17:52:27.103 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x22d7d90) skgpid ospid 12950 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:27.357 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:27.357 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:27.983 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:52:27.983 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 921 (0x7f431bfdbc90) [ CSSD]2013-05-23 17:52:27.983 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbc90 (RPC#921) tag(399002d) sent to node 1 [ CSSD]2013-05-23 17:52:27.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:27.984 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 921 (0x7f431bfdbc90) [ CSSD]2013-05-23 17:52:27.984 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbc90 (RPC#921) state 6, flags 0x202 [ CSSD]2013-05-23 17:52:27.984 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbc90, ret 0, client (nil) [ CSSD]2013-05-23 17:52:27.984 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 921 [ CSSD]2013-05-23 17:52:27.998 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:27.998 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:28.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:29.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:29.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:30.160 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:31.395 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:31.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:32.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:33.000 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:33.000 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:33.401 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:33.401 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:34.258 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:35.402 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:35.402 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:36.263 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:36.896 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:36.896 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015526, gin 1, member 0 [ CSSD]2013-05-23 17:52:36.896 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:52:36.896 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:52:36.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:36.902 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:52:36.902 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:52:37.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:37.408 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:38.003 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:38.003 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:38.008 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:52:38.008 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 922 (0x7f431bfdbd10) [ CSSD]2013-05-23 17:52:38.008 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbd10 (RPC#922) tag(39a002d) sent to node 1 [ CSSD]2013-05-23 17:52:38.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:38.009 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 922 (0x7f431bfdbd10) [ CSSD]2013-05-23 17:52:38.009 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbd10 (RPC#922) state 6, flags 0x202 [ CSSD]2013-05-23 17:52:38.009 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbd10, ret 0, client (nil) [ CSSD]2013-05-23 17:52:38.009 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 922 [ CSSD]2013-05-23 17:52:38.287 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:39.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:39.287 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015527, gin 1, member 0 [ CSSD]2013-05-23 17:52:39.287 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:52:39.287 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:52:39.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:39.293 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:52:39.293 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:52:39.413 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:39.413 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:40.298 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:41.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:41.465 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:42.134 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 17:52:42.135 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:52:42.136 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 17:52:42.136 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22fdb60) pid(13113) version 11:1:1:4 [ CSSD]2013-05-23 17:52:42.141 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fdb60), client(1/0x22d7da0) [ CSSD]2013-05-23 17:52:42.141 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 17:52:42.141 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7da0 with con 0x22e2f70, requested num -1 [ CSSD]2013-05-23 17:52:42.141 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:42.141 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 923 (0x7f431bfdbd90) [ CSSD]2013-05-23 17:52:42.141 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbd90 (RPC#923) tag(39b002d) sent to node 1 [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(039b002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015528, gin 1, member 0 [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2302b40/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbd90 (RPC#923) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdbd90, ret 0, client 0x22d7da0 member 0x22fc9c0 [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7da0 [ CSSD]2013-05-23 17:52:42.142 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 923 [ CSSD]2013-05-23 17:52:42.147 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7da0) [ CSSD]2013-05-23 17:52:42.147 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7da0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:42.147 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22fc9c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:42.147 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 924 (0x7f431bfdbe10) [ CSSD]2013-05-23 17:52:42.148 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbe10 (RPC#924) tag(39c002d) sent to node 1 [ CSSD]2013-05-23 17:52:42.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(039c002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:42.148 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:42.148 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbe10 (RPC#924) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:42.148 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbe10, ret 0, client 0x22d7da0 member 0x22fc9c0 [ CSSD]2013-05-23 17:52:42.148 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 924 [ CSSD]2013-05-23 17:52:42.148 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22fc9c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:42.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:42.350 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 17:52:42.350 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x2312880) skgpid ospid 13113 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:43.006 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:43.006 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:43.171 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 17:52:43.172 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:52:43.173 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 17:52:43.173 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22fdb60) proc(0x22fc9c0) pid(13215) version 11:1:1:4 [ CSSD]2013-05-23 17:52:43.179 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22fc9c0), client(1/0x22e2f80) [ CSSD]2013-05-23 17:52:43.179 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:52:43.179 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e2f80 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 17:52:43.179 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:43.179 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 925 (0x7f431bfdbe90) [ CSSD]2013-05-23 17:52:43.179 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbe90 (RPC#925) tag(39d002d) sent to node 1 [ CSSD]2013-05-23 17:52:43.179 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(039d002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:43.179 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015529, gin 1, member 0 [ CSSD]2013-05-23 17:52:43.179 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2319290/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:52:43.180 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:43.180 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:52:43.180 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbe90 (RPC#925) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:43.180 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdbe90, ret 0, client 0x22e2f80 member 0x22d7a00 [ CSSD]2013-05-23 17:52:43.180 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e2f80 [ CSSD]2013-05-23 17:52:43.180 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 925 [ CSSD]2013-05-23 17:52:43.185 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e2f80) [ CSSD]2013-05-23 17:52:43.185 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e2f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:43.185 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:43.185 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 926 (0x7f431bfdbf10) [ CSSD]2013-05-23 17:52:43.185 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbf10 (RPC#926) tag(39e002d) sent to node 1 [ CSSD]2013-05-23 17:52:43.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(039e002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:43.185 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:43.185 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbf10 (RPC#926) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:43.185 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdbf10, ret 0, client 0x22e2f80 member 0x22d7a00 [ CSSD]2013-05-23 17:52:43.186 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 926 [ CSSD]2013-05-23 17:52:43.186 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:43.388 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 17:52:43.388 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x22fdb60) skgpid ospid 13215 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:43.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:43.492 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:44.359 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:45.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:45.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:46.364 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:47.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:47.509 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:48.009 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:48.009 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:48.034 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:52:48.034 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 927 (0x7f431bfdbf90) [ CSSD]2013-05-23 17:52:48.035 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdbf90 (RPC#927) tag(39f002d) sent to node 1 [ CSSD]2013-05-23 17:52:48.035 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:48.035 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 927 (0x7f431bfdbf90) [ CSSD]2013-05-23 17:52:48.035 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdbf90 (RPC#927) state 6, flags 0x202 [ CSSD]2013-05-23 17:52:48.035 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdbf90, ret 0, client (nil) [ CSSD]2013-05-23 17:52:48.035 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 927 [ CSSD]2013-05-23 17:52:48.376 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:49.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:49.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:50.381 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:50.667 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c130b40) [ CSSD]2013-05-23 17:52:50.667 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c130b40), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:52:50.667 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c130b40), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:52:50.667 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:52:50.677 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:52:50.677 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:52:50.677 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c135120), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:52:50.677 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 17:52:50.678 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c082370 [ CSSD]2013-05-23 17:52:50.678 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c082370) con(0x7f431c0cf710) skgpid 7460 ospid 7460 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:51.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:51.586 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:52.140 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:52.140 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015530, gin 1, member 0 [ CSSD]2013-05-23 17:52:52.140 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:52:52.140 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:52:52.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:52.147 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:52:52.147 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:52:52.405 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:53.011 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:53.012 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:53.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:53.592 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:54.458 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:55.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:55.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:56.482 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:57.380 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:52:57.382 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:52:57.382 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:52:57.382 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c1355f0) pid(13382) version 11:1:1:4 [ CSSD]2013-05-23 17:52:57.388 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c1355f0), client(1/0x22d7cd0) [ CSSD]2013-05-23 17:52:57.388 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22d7cd0) [ CSSD]2013-05-23 17:52:57.388 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22d7cd0 with con 0x2317510, requested num -1 [ CSSD]2013-05-23 17:52:57.388 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:57.388 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 928 (0x7f431bfdc010) [ CSSD]2013-05-23 17:52:57.388 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc010 (RPC#928) tag(3a0002d) sent to node 1 [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03a0002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015531, gin 1, member 0 [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc010 (RPC#928) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc010, ret 0, client 0x22d7cd0 member 0x22e2eb0 [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22d7cd0 [ CSSD]2013-05-23 17:52:57.389 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 928 [ CSSD]2013-05-23 17:52:57.394 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22d7cd0) [ CSSD]2013-05-23 17:52:57.394 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22d7cd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:57.394 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e2eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:52:57.394 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 929 (0x7f431bfdc090) [ CSSD]2013-05-23 17:52:57.394 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc090 (RPC#929) tag(3a1002d) sent to node 1 [ CSSD]2013-05-23 17:52:57.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03a1002d) incarnation(174631051) [ CSSD]2013-05-23 17:52:57.394 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:52:57.394 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc090 (RPC#929) state 6, flags 0x100 [ CSSD]2013-05-23 17:52:57.394 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc090, ret 0, client 0x22d7cd0 member 0x22e2eb0 [ CSSD]2013-05-23 17:52:57.395 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 929 [ CSSD]2013-05-23 17:52:57.395 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e2eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:52:57.597 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:52:57.597 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c12f0c0) skgpid ospid 13382 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:52:57.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:57.608 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:52:58.015 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:52:58.015 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:52:58.058 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:52:58.058 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 930 (0x7f431bfdc110) [ CSSD]2013-05-23 17:52:58.058 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc110 (RPC#930) tag(3a2002d) sent to node 1 [ CSSD]2013-05-23 17:52:58.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:58.059 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 930 (0x7f431bfdc110) [ CSSD]2013-05-23 17:52:58.059 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc110 (RPC#930) state 6, flags 0x202 [ CSSD]2013-05-23 17:52:58.059 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc110, ret 0, client (nil) [ CSSD]2013-05-23 17:52:58.059 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 930 [ CSSD]2013-05-23 17:52:58.495 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:52:59.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:52:59.614 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:00.502 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:01.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:01.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:02.017 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:02.017 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:53:02.508 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:03.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:03.672 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:04.529 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:05.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:05.683 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:06.535 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:07.020 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:07.020 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:07.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:07.400 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015532, gin 1, member 0 [ CSSD]2013-05-23 17:53:07.400 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:53:07.400 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:53:07.407 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:07.407 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:53:07.407 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:53:07.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:07.729 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:08.084 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:53:08.085 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 931 (0x7f431bfdc190) [ CSSD]2013-05-23 17:53:08.085 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc190 (RPC#931) tag(3a3002d) sent to node 1 [ CSSD]2013-05-23 17:53:08.085 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:08.085 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 931 (0x7f431bfdc190) [ CSSD]2013-05-23 17:53:08.085 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc190 (RPC#931) state 6, flags 0x202 [ CSSD]2013-05-23 17:53:08.085 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc190, ret 0, client (nil) [ CSSD]2013-05-23 17:53:08.085 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 931 [ CSSD]2013-05-23 17:53:08.567 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:09.310 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1355f0) allocated [ CSSD]2013-05-23 17:53:09.311 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:53:09.312 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1355f0 - 1,2,3 [ CSSD]2013-05-23 17:53:09.312 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1355f0) pid(12082/12082) version 11:1:1:4 [ CSSD]2013-05-23 17:53:09.328 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c1355f0), client(1/0x22d7cd0) [ CSSD]2013-05-23 17:53:09.328 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22d7cd0) [ CSSD]2013-05-23 17:53:09.328 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:53:09.328 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 17:53:09.630 [1191196992] >TRACE: clssgmAllocProc: (0x22dfd50) allocated [ CSSD]2013-05-23 17:53:09.631 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:53:09.632 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dfd50 - 1,2,3 [ CSSD]2013-05-23 17:53:09.632 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x22dfd50) pid(13281/13281) version 11:1:1:4 [ CSSD]2013-05-23 17:53:09.636 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22dfd50), client(1/0x2312950) [ CSSD]2013-05-23 17:53:09.636 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 17:53:09.636 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:53:09.636 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:53:09.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:09.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:09.930 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22dfd50), client(2/0x205e370) [ CSSD]2013-05-23 17:53:09.930 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x205e370) [ CSSD]2013-05-23 17:53:09.930 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:53:09.930 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:53:10.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:11.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:11.767 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:12.022 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:12.022 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:12.579 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:12.628 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:53:12.629 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:53:12.629 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:53:12.630 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7410) proc(0x22d6cf0) pid(13536) version 11:1:1:4 [ CSSD]2013-05-23 17:53:12.635 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d6cf0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:53:12.635 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:12.635 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d6ef0, requested num -1 [ CSSD]2013-05-23 17:53:12.635 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:12.635 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 932 (0x7f431bfdc210) [ CSSD]2013-05-23 17:53:12.635 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc210 (RPC#932) tag(3a4002d) sent to node 1 [ CSSD]2013-05-23 17:53:12.635 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03a4002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:12.635 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015533, gin 1, member 0 [ CSSD]2013-05-23 17:53:12.635 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f39a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:53:12.636 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:53:12.636 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:53:12.636 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc210 (RPC#932) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:12.636 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc210, ret 0, client 0x22f1d30 member 0x23029d0 [ CSSD]2013-05-23 17:53:12.636 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:53:12.636 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 932 [ CSSD]2013-05-23 17:53:12.641 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:12.641 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:12.641 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23029d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:12.641 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 933 (0x7f431bfdc290) [ CSSD]2013-05-23 17:53:12.641 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc290 (RPC#933) tag(3a5002d) sent to node 1 [ CSSD]2013-05-23 17:53:12.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03a5002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:12.642 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:12.642 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc290 (RPC#933) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:12.642 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc290, ret 0, client 0x22f1d30 member 0x23029d0 [ CSSD]2013-05-23 17:53:12.642 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 933 [ CSSD]2013-05-23 17:53:12.642 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23029d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:53:12.844 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:53:12.844 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x22d7410) skgpid ospid 13536 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:53:13.773 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:13.773 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:14.609 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:15.784 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:15.784 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:16.636 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:17.024 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:17.025 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:17.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:17.809 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:18.109 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:53:18.109 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 934 (0x7f431bfdc310) [ CSSD]2013-05-23 17:53:18.109 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc310 (RPC#934) tag(3a6002d) sent to node 1 [ CSSD]2013-05-23 17:53:18.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:18.110 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 934 (0x7f431bfdc310) [ CSSD]2013-05-23 17:53:18.110 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc310 (RPC#934) state 6, flags 0x202 [ CSSD]2013-05-23 17:53:18.110 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc310, ret 0, client (nil) [ CSSD]2013-05-23 17:53:18.110 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 934 [ CSSD]2013-05-23 17:53:18.668 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:19.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:19.862 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:20.673 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:21.868 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:21.868 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:22.028 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:22.028 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:22.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:22.651 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015534, gin 1, member 0 [ CSSD]2013-05-23 17:53:22.651 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:53:22.651 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:53:22.659 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:22.659 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:53:22.659 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205e170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:53:22.679 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:23.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:23.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:24.690 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:25.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:25.885 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:26.030 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:26.030 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:53:26.737 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:27.875 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:53:27.876 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:53:27.877 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:53:27.877 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7410) proc(0x22d6cf0) pid(13670) version 11:1:1:4 [ CSSD]2013-05-23 17:53:27.882 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d6cf0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:53:27.882 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:27.882 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d6ef0, requested num -1 [ CSSD]2013-05-23 17:53:27.882 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:27.882 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 935 (0x7f431bfdc390) [ CSSD]2013-05-23 17:53:27.882 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc390 (RPC#935) tag(3a7002d) sent to node 1 [ CSSD]2013-05-23 17:53:27.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03a7002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:27.882 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015535, gin 1, member 0 [ CSSD]2013-05-23 17:53:27.882 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f39a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:53:27.883 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:53:27.883 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:53:27.883 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc390 (RPC#935) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:27.883 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc390, ret 0, client 0x22f1d30 member 0x23029d0 [ CSSD]2013-05-23 17:53:27.883 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:53:27.883 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 935 [ CSSD]2013-05-23 17:53:27.888 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:27.888 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:27.888 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23029d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:27.888 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 936 (0x7f431bfdc410) [ CSSD]2013-05-23 17:53:27.888 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc410 (RPC#936) tag(3a8002d) sent to node 1 [ CSSD]2013-05-23 17:53:27.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03a8002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:27.889 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:27.889 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc410 (RPC#936) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:27.889 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc410, ret 0, client 0x22f1d30 member 0x23029d0 [ CSSD]2013-05-23 17:53:27.889 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 936 [ CSSD]2013-05-23 17:53:27.889 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23029d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:53:27.930 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:27.930 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:28.091 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:53:28.091 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x22d7410) skgpid ospid 13670 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:53:28.132 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:53:28.133 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 937 (0x7f431bfdc490) [ CSSD]2013-05-23 17:53:28.133 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc490 (RPC#937) tag(3a9002d) sent to node 1 [ CSSD]2013-05-23 17:53:28.133 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:28.133 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 937 (0x7f431bfdc490) [ CSSD]2013-05-23 17:53:28.133 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc490 (RPC#937) state 6, flags 0x202 [ CSSD]2013-05-23 17:53:28.133 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc490, ret 0, client (nil) [ CSSD]2013-05-23 17:53:28.134 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 937 [ CSSD]2013-05-23 17:53:28.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:29.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:29.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:30.794 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:31.033 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:31.033 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:31.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:31.927 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:32.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:33.953 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:33.953 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:34.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:35.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:35.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:36.036 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:36.036 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:36.816 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:37.903 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:37.903 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015536, gin 1, member 0 [ CSSD]2013-05-23 17:53:37.903 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:53:37.903 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:53:37.909 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:37.909 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:53:37.909 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205e170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:53:37.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:37.970 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:38.160 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:53:38.160 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 938 (0x7f431bfdc510) [ CSSD]2013-05-23 17:53:38.160 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc510 (RPC#938) tag(3aa002d) sent to node 1 [ CSSD]2013-05-23 17:53:38.161 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:38.161 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 938 (0x7f431bfdc510) [ CSSD]2013-05-23 17:53:38.161 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc510 (RPC#938) state 6, flags 0x202 [ CSSD]2013-05-23 17:53:38.161 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc510, ret 0, client (nil) [ CSSD]2013-05-23 17:53:38.161 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 938 [ CSSD]2013-05-23 17:53:38.868 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:39.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:39.553 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015537, gin 1, member 0 [ CSSD]2013-05-23 17:53:39.553 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:53:39.553 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:53:39.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:39.560 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:53:39.560 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205e170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:53:39.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:39.983 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:40.875 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:41.039 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:41.039 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:41.988 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:41.988 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:42.881 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:43.123 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:53:43.124 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:53:43.125 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:53:43.125 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7410) proc(0x22d6cf0) pid(13819) version 11:1:1:4 [ CSSD]2013-05-23 17:53:43.130 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d6cf0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:53:43.130 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:43.130 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d6ef0, requested num -1 [ CSSD]2013-05-23 17:53:43.130 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:43.130 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 939 (0x7f431bfdc590) [ CSSD]2013-05-23 17:53:43.130 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc590 (RPC#939) tag(3ab002d) sent to node 1 [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ab002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015538, gin 1, member 0 [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f39a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc590 (RPC#939) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc590, ret 0, client 0x22f1d30 member 0x23029d0 [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:53:43.131 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 939 [ CSSD]2013-05-23 17:53:43.137 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:43.137 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:43.137 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23029d0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:43.137 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 940 (0x7f431bfdc610) [ CSSD]2013-05-23 17:53:43.137 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc610 (RPC#940) tag(3ac002d) sent to node 1 [ CSSD]2013-05-23 17:53:43.137 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ac002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:43.137 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:43.137 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc610 (RPC#940) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:43.137 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc610, ret 0, client 0x22f1d30 member 0x23029d0 [ CSSD]2013-05-23 17:53:43.138 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 940 [ CSSD]2013-05-23 17:53:43.138 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23029d0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:53:43.340 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:53:43.340 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x22d7410) skgpid ospid 13819 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:53:43.424 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:53:43.425 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:53:43.426 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:53:43.426 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e410) proc(0x22d6cf0) pid(13862) version 11:1:1:4 [ CSSD]2013-05-23 17:53:43.431 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22d6cf0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:53:43.431 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:43.431 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x23029a0, requested num -1 [ CSSD]2013-05-23 17:53:43.431 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:43.431 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 941 (0x7f431bfdc690) [ CSSD]2013-05-23 17:53:43.431 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc690 (RPC#941) tag(3ad002d) sent to node 1 [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ad002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015539, gin 1, member 0 [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3870/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc690 (RPC#941) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc690, ret 0, client 0x22f1d30 member 0x22f3b70 [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:53:43.432 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 941 [ CSSD]2013-05-23 17:53:43.438 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:43.438 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:43.438 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3b70) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:43.438 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 942 (0x7f431bfdc710) [ CSSD]2013-05-23 17:53:43.438 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc710 (RPC#942) tag(3ae002d) sent to node 1 [ CSSD]2013-05-23 17:53:43.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ae002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:43.438 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:43.439 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc710 (RPC#942) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:43.439 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc710, ret 0, client 0x22f1d30 member 0x22f3b70 [ CSSD]2013-05-23 17:53:43.439 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 942 [ CSSD]2013-05-23 17:53:43.439 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3b70) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:53:43.641 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:53:43.641 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x205e410) skgpid ospid 13862 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:53:44.014 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:44.014 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:44.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:44.062 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015540, gin 1, member 0 [ CSSD]2013-05-23 17:53:44.062 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22d7350/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:53:44.062 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:53:44.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:44.068 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:53:44.068 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x205e170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:53:44.192 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:53:44.193 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:53:44.194 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:53:44.194 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:53:44.196 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:53:44.196 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:53:44.891 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:46.042 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:46.042 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:46.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:46.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:46.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:48.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:48.110 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:48.184 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:53:48.184 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 943 (0x7f431bfdc790) [ CSSD]2013-05-23 17:53:48.185 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc790 (RPC#943) tag(3af002d) sent to node 1 [ CSSD]2013-05-23 17:53:48.185 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:48.185 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 943 (0x7f431bfdc790) [ CSSD]2013-05-23 17:53:48.185 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc790 (RPC#943) state 6, flags 0x202 [ CSSD]2013-05-23 17:53:48.185 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc790, ret 0, client (nil) [ CSSD]2013-05-23 17:53:48.185 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 943 [ CSSD]2013-05-23 17:53:48.909 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:50.044 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:50.044 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:53:50.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:50.163 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:50.915 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:52.169 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:52.169 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:52.921 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:53.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:53.152 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015541, gin 1, member 0 [ CSSD]2013-05-23 17:53:53.152 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:53:53.152 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:53:53.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:53.159 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:53:53.159 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205e170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:53:54.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:54.175 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:54.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:55.047 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:53:55.047 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:53:55.265 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c137c40) [ CSSD]2013-05-23 17:53:55.265 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c137c40), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:53:55.265 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c137c40), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:53:55.265 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:53:55.269 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c1375c0) [ CSSD]2013-05-23 17:53:55.269 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c1375c0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:53:55.269 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c1375c0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:53:55.269 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 17:53:55.269 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c137120 [ CSSD]2013-05-23 17:53:55.269 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c137120) con(0x7f431c090cf0) skgpid 8171 ospid 8171 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:53:56.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:56.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:56.997 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:53:58.208 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:53:58.208 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 944 (0x7f431bfdc810) [ CSSD]2013-05-23 17:53:58.208 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc810 (RPC#944) tag(3b0002d) sent to node 1 [ CSSD]2013-05-23 17:53:58.209 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:58.209 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 944 (0x7f431bfdc810) [ CSSD]2013-05-23 17:53:58.209 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc810 (RPC#944) state 6, flags 0x202 [ CSSD]2013-05-23 17:53:58.209 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc810, ret 0, client (nil) [ CSSD]2013-05-23 17:53:58.209 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 944 [ CSSD]2013-05-23 17:53:58.211 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:53:58.211 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:53:58.373 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:53:58.374 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:53:58.375 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:53:58.375 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e410) proc(0x22d6cf0) pid(14003) version 11:1:1:4 [ CSSD]2013-05-23 17:53:58.380 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:53:58.380 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:58.380 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x23029a0, requested num -1 [ CSSD]2013-05-23 17:53:58.380 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:58.380 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 945 (0x7f431bfdc890) [ CSSD]2013-05-23 17:53:58.380 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc890 (RPC#945) tag(3b1002d) sent to node 1 [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03b1002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015542, gin 1, member 0 [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3870/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc890 (RPC#945) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdc890, ret 0, client 0x22f1d30 member 0x22f3b70 [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 17:53:58.381 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 945 [ CSSD]2013-05-23 17:53:58.387 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:53:58.387 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:58.387 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3b70) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:53:58.387 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 946 (0x7f431bfdc910) [ CSSD]2013-05-23 17:53:58.387 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc910 (RPC#946) tag(3b2002d) sent to node 1 [ CSSD]2013-05-23 17:53:58.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03b2002d) incarnation(174631051) [ CSSD]2013-05-23 17:53:58.388 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:53:58.388 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc910 (RPC#946) state 6, flags 0x100 [ CSSD]2013-05-23 17:53:58.388 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdc910, ret 0, client 0x22f1d30 member 0x22f3b70 [ CSSD]2013-05-23 17:53:58.388 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 946 [ CSSD]2013-05-23 17:53:58.388 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3b70) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:53:58.590 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 17:53:58.590 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x205e410) skgpid ospid 14003 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:53:59.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:00.049 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:00.049 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:00.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:00.214 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:01.014 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:02.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:02.219 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:03.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:04.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:04.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:05.052 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:05.052 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:05.072 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:06.277 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:06.277 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:07.097 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:08.254 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:54:08.255 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 947 (0x7f431bfdc990) [ CSSD]2013-05-23 17:54:08.255 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdc990 (RPC#947) tag(3b3002d) sent to node 1 [ CSSD]2013-05-23 17:54:08.255 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:08.255 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 947 (0x7f431bfdc990) [ CSSD]2013-05-23 17:54:08.255 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdc990 (RPC#947) state 6, flags 0x202 [ CSSD]2013-05-23 17:54:08.255 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdc990, ret 0, client (nil) [ CSSD]2013-05-23 17:54:08.255 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 947 [ CSSD]2013-05-23 17:54:08.283 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:08.283 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:08.414 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:08.414 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015543, gin 1, member 0 [ CSSD]2013-05-23 17:54:08.414 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:54:08.414 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:54:08.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:08.422 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:54:08.422 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205e170) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:54:09.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:10.005 [1191196992] >TRACE: clssgmAllocProc: (0x22d6cf0) allocated [ CSSD]2013-05-23 17:54:10.006 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:54:10.006 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6cf0 - 1,2,3 [ CSSD]2013-05-23 17:54:10.006 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7410) proc(0x22d6cf0) pid(13980/13980) version 11:1:1:4 [ CSSD]2013-05-23 17:54:10.022 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(1/0x22f1d30) [ CSSD]2013-05-23 17:54:10.023 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 17:54:10.023 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:54:10.023 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 31 [ CSSD]2013-05-23 17:54:10.055 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:10.055 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:10.294 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:10.294 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:10.363 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6cf0), client(2/0x22d7650) [ CSSD]2013-05-23 17:54:10.363 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22d7650) [ CSSD]2013-05-23 17:54:10.363 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:54:10.363 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:54:11.114 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:12.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:12.340 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:13.119 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:13.622 [1191196992] >TRACE: clssgmAllocProc: (0x22f3050) allocated [ CSSD]2013-05-23 17:54:13.623 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:54:13.624 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3050 - 1,2,3 [ CSSD]2013-05-23 17:54:13.624 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3da0) proc(0x22f3050) pid(14144) version 11:1:1:4 [ CSSD]2013-05-23 17:54:13.629 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f3050), client(1/0x22f50d0) [ CSSD]2013-05-23 17:54:13.629 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f50d0) [ CSSD]2013-05-23 17:54:13.629 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f50d0 with con 0x22f4bd0, requested num -1 [ CSSD]2013-05-23 17:54:13.629 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:13.629 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 948 (0x7f431bfdca10) [ CSSD]2013-05-23 17:54:13.630 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdca10 (RPC#948) tag(3b4002d) sent to node 1 [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03b4002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015544, gin 1, member 0 [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23181f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdca10 (RPC#948) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdca10, ret 0, client 0x22f50d0 member 0x22f2e50 [ CSSD]2013-05-23 17:54:13.630 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f50d0 [ CSSD]2013-05-23 17:54:13.631 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 948 [ CSSD]2013-05-23 17:54:13.636 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f50d0) [ CSSD]2013-05-23 17:54:13.636 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f50d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:13.636 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:13.636 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 949 (0x7f431bfdca90) [ CSSD]2013-05-23 17:54:13.636 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdca90 (RPC#949) tag(3b5002d) sent to node 1 [ CSSD]2013-05-23 17:54:13.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03b5002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:13.637 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:13.637 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdca90 (RPC#949) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:13.637 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdca90, ret 0, client 0x22f50d0 member 0x22f2e50 [ CSSD]2013-05-23 17:54:13.637 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 949 [ CSSD]2013-05-23 17:54:13.637 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:54:13.839 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3050 [ CSSD]2013-05-23 17:54:13.839 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3050) con(0x22f3da0) skgpid ospid 14144 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:54:14.058 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:14.058 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:54:14.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:14.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:15.173 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:16.377 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:16.377 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:17.178 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:18.280 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:54:18.280 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 950 (0x7f431bfdcb10) [ CSSD]2013-05-23 17:54:18.280 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcb10 (RPC#950) tag(3b6002d) sent to node 1 [ CSSD]2013-05-23 17:54:18.281 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:18.281 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 950 (0x7f431bfdcb10) [ CSSD]2013-05-23 17:54:18.281 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcb10 (RPC#950) state 6, flags 0x202 [ CSSD]2013-05-23 17:54:18.281 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcb10, ret 0, client (nil) [ CSSD]2013-05-23 17:54:18.281 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 950 [ CSSD]2013-05-23 17:54:18.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:18.383 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:19.061 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:19.061 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:19.190 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:20.394 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:20.395 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:21.215 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:22.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:22.400 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:23.221 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:23.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:23.666 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015545, gin 1, member 0 [ CSSD]2013-05-23 17:54:23.666 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3c10/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:54:23.666 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:54:23.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:23.673 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:54:23.673 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f3960) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:54:24.064 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:24.064 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:24.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:24.426 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:25.223 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:26.497 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:26.497 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:27.247 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:28.329 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:54:28.329 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 951 (0x7f431bfdcb90) [ CSSD]2013-05-23 17:54:28.329 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcb90 (RPC#951) tag(3b7002d) sent to node 1 [ CSSD]2013-05-23 17:54:28.330 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:28.330 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 951 (0x7f431bfdcb90) [ CSSD]2013-05-23 17:54:28.330 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcb90 (RPC#951) state 6, flags 0x202 [ CSSD]2013-05-23 17:54:28.330 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcb90, ret 0, client (nil) [ CSSD]2013-05-23 17:54:28.330 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 951 [ CSSD]2013-05-23 17:54:28.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:28.503 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:28.871 [1191196992] >TRACE: clssgmAllocProc: (0x22f2fa0) allocated [ CSSD]2013-05-23 17:54:28.872 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:54:28.872 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2fa0 - 1,2,3 [ CSSD]2013-05-23 17:54:28.872 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2120) proc(0x22f2fa0) pid(14293) version 11:1:1:4 [ CSSD]2013-05-23 17:54:28.889 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f2fa0), client(1/0x22f4f80) [ CSSD]2013-05-23 17:54:28.889 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4f80) [ CSSD]2013-05-23 17:54:28.889 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4f80 with con 0x22f4a80, requested num -1 [ CSSD]2013-05-23 17:54:28.889 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:28.889 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 952 (0x7f431bfdcc10) [ CSSD]2013-05-23 17:54:28.889 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcc10 (RPC#952) tag(3b8002d) sent to node 1 [ CSSD]2013-05-23 17:54:28.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03b8002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:28.889 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015546, gin 1, member 0 [ CSSD]2013-05-23 17:54:28.889 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23181f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:54:28.890 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:54:28.890 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:54:28.890 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcc10 (RPC#952) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:28.890 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdcc10, ret 0, client 0x22f4f80 member 0x22f2e90 [ CSSD]2013-05-23 17:54:28.890 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4f80 [ CSSD]2013-05-23 17:54:28.890 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 952 [ CSSD]2013-05-23 17:54:28.897 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4f80) [ CSSD]2013-05-23 17:54:28.897 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:28.897 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e90) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:28.897 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 953 (0x7f431bfdcc90) [ CSSD]2013-05-23 17:54:28.897 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcc90 (RPC#953) tag(3b9002d) sent to node 1 [ CSSD]2013-05-23 17:54:28.897 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03b9002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:28.897 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:28.897 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcc90 (RPC#953) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:28.897 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdcc90, ret 0, client 0x22f4f80 member 0x22f2e90 [ CSSD]2013-05-23 17:54:28.897 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 953 [ CSSD]2013-05-23 17:54:28.898 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e90) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:54:29.067 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:29.067 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:29.100 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2fa0 [ CSSD]2013-05-23 17:54:29.100 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2fa0) con(0x22f2120) skgpid ospid 14293 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:54:29.281 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:30.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:30.515 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:31.287 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:31.398 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:54:31.398 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:54:31.399 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:54:31.399 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:54:31.572 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:31.572 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015547, gin 1, member 0 [ CSSD]2013-05-23 17:54:31.572 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22f3bf0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:54:31.572 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:54:31.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:31.579 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:54:31.579 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22f3960) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:54:32.521 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:32.521 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:33.293 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:34.070 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:34.070 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:34.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:34.527 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:35.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:36.578 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:36.579 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:37.328 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:38.072 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:38.072 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:54:38.355 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:54:38.355 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 954 (0x7f431bfdcd10) [ CSSD]2013-05-23 17:54:38.355 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcd10 (RPC#954) tag(3ba002d) sent to node 1 [ CSSD]2013-05-23 17:54:38.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:38.356 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 954 (0x7f431bfdcd10) [ CSSD]2013-05-23 17:54:38.356 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcd10 (RPC#954) state 6, flags 0x202 [ CSSD]2013-05-23 17:54:38.356 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcd10, ret 0, client (nil) [ CSSD]2013-05-23 17:54:38.356 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 954 [ CSSD]2013-05-23 17:54:38.604 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:38.604 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:38.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:38.921 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015548, gin 1, member 0 [ CSSD]2013-05-23 17:54:38.921 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:54:38.921 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:54:38.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:38.928 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:54:38.928 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f3870) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:54:39.401 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:39.811 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:39.811 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015549, gin 1, member 0 [ CSSD]2013-05-23 17:54:39.811 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3c10/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:54:39.811 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:54:39.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:39.817 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:54:39.817 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f3960) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:54:40.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:40.616 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:41.406 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:42.622 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:42.622 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:43.075 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:43.075 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:43.418 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:43.682 [1191196992] >TRACE: clssgmAllocProc: (0x22f3050) allocated [ CSSD]2013-05-23 17:54:43.682 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:54:43.683 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3050 - 1,2,3 [ CSSD]2013-05-23 17:54:43.683 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3dc0) proc(0x22f3050) pid(14443) version 11:1:1:4 [ CSSD]2013-05-23 17:54:43.688 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f3050), client(1/0x22f50b0) [ CSSD]2013-05-23 17:54:43.688 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f50b0) [ CSSD]2013-05-23 17:54:43.688 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f50b0 with con 0x22f4bb0, requested num -1 [ CSSD]2013-05-23 17:54:43.688 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:43.688 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 955 (0x7f431bfdcd90) [ CSSD]2013-05-23 17:54:43.689 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcd90 (RPC#955) tag(3bb002d) sent to node 1 [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03bb002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015550, gin 1, member 0 [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23181f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcd90 (RPC#955) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdcd90, ret 0, client 0x22f50b0 member 0x22f2e50 [ CSSD]2013-05-23 17:54:43.689 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f50b0 [ CSSD]2013-05-23 17:54:43.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 955 [ CSSD]2013-05-23 17:54:43.695 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f50b0) [ CSSD]2013-05-23 17:54:43.695 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f50b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:43.695 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:43.695 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 956 (0x7f431bfdce10) [ CSSD]2013-05-23 17:54:43.696 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdce10 (RPC#956) tag(3bc002d) sent to node 1 [ CSSD]2013-05-23 17:54:43.696 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03bc002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:43.696 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:43.696 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdce10 (RPC#956) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:43.696 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdce10, ret 0, client 0x22f50b0 member 0x22f2e50 [ CSSD]2013-05-23 17:54:43.696 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 956 [ CSSD]2013-05-23 17:54:43.696 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:54:43.899 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3050 [ CSSD]2013-05-23 17:54:43.899 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3050) con(0x22f3dc0) skgpid ospid 14443 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:54:44.139 [1191196992] >TRACE: clssgmAllocProc: (0x22f2f90) allocated [ CSSD]2013-05-23 17:54:44.140 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:54:44.141 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f2f90 - 1,2,3 [ CSSD]2013-05-23 17:54:44.141 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3b40) proc(0x22f2f90) pid(14458) version 11:1:1:4 [ CSSD]2013-05-23 17:54:44.146 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f2f90), client(1/0x22f4f80) [ CSSD]2013-05-23 17:54:44.146 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4f80) [ CSSD]2013-05-23 17:54:44.146 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4f80 with con 0x22f4a80, requested num -1 [ CSSD]2013-05-23 17:54:44.146 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:44.146 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 957 (0x7f431bfdce90) [ CSSD]2013-05-23 17:54:44.147 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdce90 (RPC#957) tag(3bd002d) sent to node 1 [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03bd002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015551, gin 1, member 0 [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23181f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdce90 (RPC#957) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdce90, ret 0, client 0x22f4f80 member 0x22f2e80 [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4f80 [ CSSD]2013-05-23 17:54:44.147 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 957 [ CSSD]2013-05-23 17:54:44.153 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4f80) [ CSSD]2013-05-23 17:54:44.153 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:44.153 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e80) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:44.153 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 958 (0x7f431bfdcf10) [ CSSD]2013-05-23 17:54:44.153 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcf10 (RPC#958) tag(3be002d) sent to node 1 [ CSSD]2013-05-23 17:54:44.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03be002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:44.153 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:44.153 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcf10 (RPC#958) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:44.153 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdcf10, ret 0, client 0x22f4f80 member 0x22f2e80 [ CSSD]2013-05-23 17:54:44.154 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 958 [ CSSD]2013-05-23 17:54:44.154 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:54:44.356 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f2f90 [ CSSD]2013-05-23 17:54:44.356 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f2f90) con(0x22f3b40) skgpid ospid 14458 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:54:44.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:44.628 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:45.424 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:46.679 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:46.679 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:47.430 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:48.078 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:48.078 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:48.385 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:54:48.386 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 959 (0x7f431bfdcf90) [ CSSD]2013-05-23 17:54:48.386 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdcf90 (RPC#959) tag(3bf002d) sent to node 1 [ CSSD]2013-05-23 17:54:48.386 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:48.386 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 959 (0x7f431bfdcf90) [ CSSD]2013-05-23 17:54:48.386 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdcf90 (RPC#959) state 6, flags 0x202 [ CSSD]2013-05-23 17:54:48.386 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdcf90, ret 0, client (nil) [ CSSD]2013-05-23 17:54:48.387 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 959 [ CSSD]2013-05-23 17:54:48.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:48.725 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:49.481 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:50.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:50.796 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:51.486 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:52.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:52.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:53.081 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:53.081 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:53.580 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:54.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:54.171 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015552, gin 1, member 0 [ CSSD]2013-05-23 17:54:54.171 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3c10/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:54:54.171 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:54:54.178 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:54.178 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:54:54.178 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f3960) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:54:54.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:54.808 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:55.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:56.818 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:56.818 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:57.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:54:58.083 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:54:58.083 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:54:58.416 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:54:58.416 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 960 (0x7f431bfdd010) [ CSSD]2013-05-23 17:54:58.416 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd010 (RPC#960) tag(3c0002d) sent to node 1 [ CSSD]2013-05-23 17:54:58.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:58.417 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 960 (0x7f431bfdd010) [ CSSD]2013-05-23 17:54:58.417 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd010 (RPC#960) state 6, flags 0x202 [ CSSD]2013-05-23 17:54:58.417 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd010, ret 0, client (nil) [ CSSD]2013-05-23 17:54:58.417 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 960 [ CSSD]2013-05-23 17:54:58.824 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:54:58.824 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:54:59.388 [1191196992] >TRACE: clssgmAllocProc: (0x22f3020) allocated [ CSSD]2013-05-23 17:54:59.389 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:54:59.389 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3020 - 1,2,3 [ CSSD]2013-05-23 17:54:59.389 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2120) proc(0x22f3020) pid(14601) version 11:1:1:4 [ CSSD]2013-05-23 17:54:59.406 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f3020), client(1/0x22f5000) [ CSSD]2013-05-23 17:54:59.406 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f5000) [ CSSD]2013-05-23 17:54:59.406 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f5000 with con 0x22f2f40, requested num -1 [ CSSD]2013-05-23 17:54:59.406 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:59.406 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 961 (0x7f431bfdd090) [ CSSD]2013-05-23 17:54:59.406 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd090 (RPC#961) tag(3c1002d) sent to node 1 [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03c1002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015553, gin 1, member 0 [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23181f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd090 (RPC#961) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd090, ret 0, client 0x22f5000 member 0x22f5170 [ CSSD]2013-05-23 17:54:59.406 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f5000 [ CSSD]2013-05-23 17:54:59.407 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 961 [ CSSD]2013-05-23 17:54:59.412 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f5000) [ CSSD]2013-05-23 17:54:59.412 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f5000), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:59.412 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f5170) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:54:59.412 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 962 (0x7f431bfdd110) [ CSSD]2013-05-23 17:54:59.412 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd110 (RPC#962) tag(3c2002d) sent to node 1 [ CSSD]2013-05-23 17:54:59.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03c2002d) incarnation(174631051) [ CSSD]2013-05-23 17:54:59.413 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:54:59.413 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd110 (RPC#962) state 6, flags 0x100 [ CSSD]2013-05-23 17:54:59.413 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd110, ret 0, client 0x22f5000 member 0x22f5170 [ CSSD]2013-05-23 17:54:59.413 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 962 [ CSSD]2013-05-23 17:54:59.413 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f5170) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:54:59.615 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3020 [ CSSD]2013-05-23 17:54:59.615 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3020) con(0x22f2120) skgpid ospid 14601 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:54:59.623 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:00.877 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:00.877 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:01.628 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:02.086 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:02.086 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:55:02.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:02.883 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:03.680 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:04.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:04.895 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:05.685 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:06.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:06.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:07.089 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:07.089 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:07.691 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:08.440 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:55:08.441 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 963 (0x7f431bfdd190) [ CSSD]2013-05-23 17:55:08.441 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd190 (RPC#963) tag(3c3002d) sent to node 1 [ CSSD]2013-05-23 17:55:08.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:08.441 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 963 (0x7f431bfdd190) [ CSSD]2013-05-23 17:55:08.441 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd190 (RPC#963) state 6, flags 0x202 [ CSSD]2013-05-23 17:55:08.441 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd190, ret 0, client (nil) [ CSSD]2013-05-23 17:55:08.442 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 963 [ CSSD]2013-05-23 17:55:08.490 [1191196992] >TRACE: clssgmAllocProc: (0x22f3000) allocated [ CSSD]2013-05-23 17:55:08.491 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:55:08.491 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3000 - 1,2,3 [ CSSD]2013-05-23 17:55:08.491 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2120) proc(0x22f3000) pid(14718) version 11:1:1:4 [ CSSD]2013-05-23 17:55:08.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:08.926 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:09.085 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3000 [ CSSD]2013-05-23 17:55:09.085 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3000) con(0x22f2120) skgpid ospid 14718 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:55:09.422 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:09.422 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015554, gin 1, member 0 [ CSSD]2013-05-23 17:55:09.422 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:55:09.422 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:55:09.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:09.431 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:55:09.431 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f3870) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:55:09.703 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:10.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:10.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:11.709 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:12.092 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:12.092 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:13.003 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:13.003 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:13.721 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:14.644 [1191196992] >TRACE: clssgmAllocProc: (0x22f3050) allocated [ CSSD]2013-05-23 17:55:14.645 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:55:14.646 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3050 - 1,2,3 [ CSSD]2013-05-23 17:55:14.646 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f3da0) proc(0x22f3050) pid(14860) version 11:1:1:4 [ CSSD]2013-05-23 17:55:14.651 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f3050), client(1/0x22f50d0) [ CSSD]2013-05-23 17:55:14.651 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f50d0) [ CSSD]2013-05-23 17:55:14.651 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f50d0 with con 0x22f4bd0, requested num -1 [ CSSD]2013-05-23 17:55:14.651 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:14.651 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 964 (0x7f431bfdd210) [ CSSD]2013-05-23 17:55:14.651 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd210 (RPC#964) tag(3c4002d) sent to node 1 [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03c4002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015555, gin 1, member 0 [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x23181f0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd210 (RPC#964) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd210, ret 0, client 0x22f50d0 member 0x22f2e50 [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f50d0 [ CSSD]2013-05-23 17:55:14.652 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 964 [ CSSD]2013-05-23 17:55:14.657 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f50d0) [ CSSD]2013-05-23 17:55:14.657 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f50d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:14.657 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f2e50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:14.657 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 965 (0x7f431bfdd290) [ CSSD]2013-05-23 17:55:14.657 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd290 (RPC#965) tag(3c5002d) sent to node 1 [ CSSD]2013-05-23 17:55:14.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03c5002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:14.658 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:14.658 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd290 (RPC#965) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:14.658 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd290, ret 0, client 0x22f50d0 member 0x22f2e50 [ CSSD]2013-05-23 17:55:14.658 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 965 [ CSSD]2013-05-23 17:55:14.658 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f2e50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:55:14.860 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3050 [ CSSD]2013-05-23 17:55:14.860 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3050) con(0x22f3da0) skgpid ospid 14860 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:55:15.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:15.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:15.746 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:17.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:17.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:17.095 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:17.095 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:17.771 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:18.467 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:55:18.467 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 966 (0x7f431bfdd310) [ CSSD]2013-05-23 17:55:18.467 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd310 (RPC#966) tag(3c6002d) sent to node 1 [ CSSD]2013-05-23 17:55:18.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:18.468 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 966 (0x7f431bfdd310) [ CSSD]2013-05-23 17:55:18.468 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd310 (RPC#966) state 6, flags 0x202 [ CSSD]2013-05-23 17:55:18.468 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd310, ret 0, client (nil) [ CSSD]2013-05-23 17:55:18.468 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 966 [ CSSD]2013-05-23 17:55:18.972 [1191196992] >TRACE: clssgmAllocProc: (0x22f3000) allocated [ CSSD]2013-05-23 17:55:18.974 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:55:18.974 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3000 - 1,2,3 [ CSSD]2013-05-23 17:55:18.974 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f2120) proc(0x22f3000) pid(13279/13279) version 11:1:1:4 [ CSSD]2013-05-23 17:55:18.980 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f3000), client(1/0x22f4db0) [ CSSD]2013-05-23 17:55:18.980 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22f4db0) [ CSSD]2013-05-23 17:55:18.980 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:55:18.980 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:55:19.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:19.047 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:19.783 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:20.697 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c14f3d0) [ CSSD]2013-05-23 17:55:20.697 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c14f3d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:55:20.697 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c14f3d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:55:20.697 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 17:55:20.702 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c137520) [ CSSD]2013-05-23 17:55:20.702 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c137520), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:55:20.702 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c137520), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:55:20.702 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 17:55:20.702 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13bc40 [ CSSD]2013-05-23 17:55:20.702 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13bc40) con(0x7f431c137470) skgpid 9500 ospid 9500 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:55:21.088 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:21.088 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:21.297 [1191196992] >TRACE: clssgmAllocProc: (0x22ed960) allocated [ CSSD]2013-05-23 17:55:21.297 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:55:21.298 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ed960 - 1,2,3 [ CSSD]2013-05-23 17:55:21.298 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22f4f20) proc(0x22ed960) pid(14956/14956) version 11:1:1:4 [ CSSD]2013-05-23 17:55:21.313 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ed960), client(1/0x22ede80) [ CSSD]2013-05-23 17:55:21.313 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ede80) [ CSSD]2013-05-23 17:55:21.313 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:55:21.313 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 17:55:21.488 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22ed960), client(2/0x22ee170) [ CSSD]2013-05-23 17:55:21.489 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22ee170) [ CSSD]2013-05-23 17:55:21.489 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:55:21.489 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 17:55:21.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:22.098 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:22.098 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:23.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:23.094 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:23.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:24.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:24.686 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015556, gin 1, member 0 [ CSSD]2013-05-23 17:55:24.686 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:55:24.686 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:55:24.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:24.694 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:55:24.694 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e6d60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:55:25.186 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:25.186 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:25.807 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:27.101 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:27.101 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:27.212 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:27.212 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:27.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:28.491 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:55:28.491 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 967 (0x7f431bfdd390) [ CSSD]2013-05-23 17:55:28.492 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd390 (RPC#967) tag(3c7002d) sent to node 1 [ CSSD]2013-05-23 17:55:28.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:28.492 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 967 (0x7f431bfdd390) [ CSSD]2013-05-23 17:55:28.492 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd390 (RPC#967) state 6, flags 0x202 [ CSSD]2013-05-23 17:55:28.492 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd390, ret 0, client (nil) [ CSSD]2013-05-23 17:55:28.492 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 967 [ CSSD]2013-05-23 17:55:29.218 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:29.218 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:29.893 [1191196992] >TRACE: clssgmAllocProc: (0x23218d0) allocated [ CSSD]2013-05-23 17:55:29.894 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:55:29.895 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23218d0 - 1,2,3 [ CSSD]2013-05-23 17:55:29.895 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e3b60) proc(0x23218d0) pid(15032) version 11:1:1:4 [ CSSD]2013-05-23 17:55:29.900 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x23218d0), client(1/0x2321fb0) [ CSSD]2013-05-23 17:55:29.900 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2321fb0) [ CSSD]2013-05-23 17:55:29.900 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2321fb0 with con 0x22e5400, requested num -1 [ CSSD]2013-05-23 17:55:29.900 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:29.900 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 968 (0x7f431bfdd410) [ CSSD]2013-05-23 17:55:29.900 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd410 (RPC#968) tag(3c8002d) sent to node 1 [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03c8002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015557, gin 1, member 0 [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2322120/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd410 (RPC#968) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd410, ret 0, client 0x2321fb0 member 0x2322420 [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2321fb0 [ CSSD]2013-05-23 17:55:29.901 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 968 [ CSSD]2013-05-23 17:55:29.904 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:29.907 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2321fb0) [ CSSD]2013-05-23 17:55:29.907 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2321fb0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:29.907 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2322420) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:29.907 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 969 (0x7f431bfdd490) [ CSSD]2013-05-23 17:55:29.907 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd490 (RPC#969) tag(3c9002d) sent to node 1 [ CSSD]2013-05-23 17:55:29.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03c9002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:29.908 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:29.908 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd490 (RPC#969) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:29.908 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd490, ret 0, client 0x2321fb0 member 0x2322420 [ CSSD]2013-05-23 17:55:29.908 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 969 [ CSSD]2013-05-23 17:55:29.908 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2322420) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:55:30.111 [1191196992] >TRACE: clssgmDeadProc: proc 0x23218d0 [ CSSD]2013-05-23 17:55:30.111 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23218d0) con(0x22e3b60) skgpid ospid 15032 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:55:31.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:31.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:31.909 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:32.104 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:32.104 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:33.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:33.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:33.921 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:35.286 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:35.286 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:35.927 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:37.107 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:37.107 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:37.292 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:37.292 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:37.933 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:38.515 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:55:38.516 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 970 (0x7f431bfdd510) [ CSSD]2013-05-23 17:55:38.516 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd510 (RPC#970) tag(3ca002d) sent to node 1 [ CSSD]2013-05-23 17:55:38.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:38.516 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 970 (0x7f431bfdd510) [ CSSD]2013-05-23 17:55:38.516 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd510 (RPC#970) state 6, flags 0x202 [ CSSD]2013-05-23 17:55:38.516 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd510, ret 0, client (nil) [ CSSD]2013-05-23 17:55:38.517 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 970 [ CSSD]2013-05-23 17:55:39.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:39.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:39.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:39.939 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015558, gin 1, member 0 [ CSSD]2013-05-23 17:55:39.939 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:55:39.939 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:55:39.945 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:39.945 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:55:39.945 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c109eb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:55:39.955 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:40.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:40.068 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015559, gin 1, member 0 [ CSSD]2013-05-23 17:55:40.068 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:55:40.068 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:55:40.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:40.107 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:55:40.107 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0cf650) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:55:41.408 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:41.408 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:41.961 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:42.006 [1191196992] >TRACE: clssgmAllocProc: (0x22e53b0) allocated [ CSSD]2013-05-23 17:55:42.006 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:55:42.007 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e53b0 - 1,2,3 [ CSSD]2013-05-23 17:55:42.007 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e3c90) proc(0x22e53b0) pid(14962/14962) version 11:1:1:4 [ CSSD]2013-05-23 17:55:42.012 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22e53b0), client(1/0x2321f80) [ CSSD]2013-05-23 17:55:42.012 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2321f80) [ CSSD]2013-05-23 17:55:42.012 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:55:42.012 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 17:55:42.110 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:42.110 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:42.516 [1191196992] >TRACE: clssgmAllocProc: (0x22e8ca0) allocated [ CSSD]2013-05-23 17:55:42.517 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:55:42.518 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22e8ca0 - 1,2,3 [ CSSD]2013-05-23 17:55:42.518 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2322680) proc(0x22e8ca0) pid(14960/14960) version 11:1:1:4 [ CSSD]2013-05-23 17:55:42.534 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8ca0), client(1/0x22e9290) [ CSSD]2013-05-23 17:55:42.534 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22e9290) [ CSSD]2013-05-23 17:55:42.534 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 17:55:42.534 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 17:55:42.887 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x22e8ca0), client(2/0x22e9790) [ CSSD]2013-05-23 17:55:42.887 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22e9790) [ CSSD]2013-05-23 17:55:42.887 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 17:55:42.887 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 17:55:43.414 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:43.414 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:43.939 [1191196992] >TRACE: clssgmAllocProc: (0x2328070) allocated [ CSSD]2013-05-23 17:55:43.940 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:55:43.941 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2328070 - 1,2,3 [ CSSD]2013-05-23 17:55:43.941 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e35d0) proc(0x2328070) pid(15173) version 11:1:1:4 [ CSSD]2013-05-23 17:55:43.957 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2328070), client(1/0x2327fd0) [ CSSD]2013-05-23 17:55:43.957 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2327fd0) [ CSSD]2013-05-23 17:55:43.957 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2327fd0 with con 0x2328360, requested num -1 [ CSSD]2013-05-23 17:55:43.957 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:43.957 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 971 (0x7f431bfdd590) [ CSSD]2013-05-23 17:55:43.958 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd590 (RPC#971) tag(3cb002d) sent to node 1 [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03cb002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015560, gin 1, member 0 [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328950/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd590 (RPC#971) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd590, ret 0, client 0x2327fd0 member 0x2328c50 [ CSSD]2013-05-23 17:55:43.958 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2327fd0 [ CSSD]2013-05-23 17:55:43.959 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 971 [ CSSD]2013-05-23 17:55:43.964 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2327fd0) [ CSSD]2013-05-23 17:55:43.965 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2327fd0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:43.965 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2328c50) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:43.965 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 972 (0x7f431bfdd610) [ CSSD]2013-05-23 17:55:43.965 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd610 (RPC#972) tag(3cc002d) sent to node 1 [ CSSD]2013-05-23 17:55:43.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03cc002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:43.965 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:43.965 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd610 (RPC#972) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:43.965 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd610, ret 0, client 0x2327fd0 member 0x2328c50 [ CSSD]2013-05-23 17:55:43.965 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 972 [ CSSD]2013-05-23 17:55:43.965 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2328c50) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:55:43.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:44.168 [1191196992] >TRACE: clssgmDeadProc: proc 0x2328070 [ CSSD]2013-05-23 17:55:44.168 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2328070) con(0x22e35d0) skgpid ospid 15173 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:55:45.148 [1191196992] >TRACE: clssgmAllocProc: (0x2327120) allocated [ CSSD]2013-05-23 17:55:45.149 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:55:45.150 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327120 - 1,2,3 [ CSSD]2013-05-23 17:55:45.150 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e35d0) proc(0x2327120) pid(15188) version 11:1:1:4 [ CSSD]2013-05-23 17:55:45.167 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2327120), client(1/0x2327080) [ CSSD]2013-05-23 17:55:45.167 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2327080) [ CSSD]2013-05-23 17:55:45.167 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2327080 with con 0x22ea390, requested num -1 [ CSSD]2013-05-23 17:55:45.167 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:45.167 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 973 (0x7f431bfdd690) [ CSSD]2013-05-23 17:55:45.167 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd690 (RPC#973) tag(3cd002d) sent to node 1 [ CSSD]2013-05-23 17:55:45.167 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03cd002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:45.167 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015561, gin 1, member 0 [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2327930/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd690 (RPC#973) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd690, ret 0, client 0x2327080 member 0x2327c30 [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2327080 [ CSSD]2013-05-23 17:55:45.168 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 973 [ CSSD]2013-05-23 17:55:45.174 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2327080) [ CSSD]2013-05-23 17:55:45.174 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2327080), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:45.174 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327c30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:55:45.174 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 974 (0x7f431bfdd710) [ CSSD]2013-05-23 17:55:45.174 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd710 (RPC#974) tag(3ce002d) sent to node 1 [ CSSD]2013-05-23 17:55:45.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ce002d) incarnation(174631051) [ CSSD]2013-05-23 17:55:45.175 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:55:45.175 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd710 (RPC#974) state 6, flags 0x100 [ CSSD]2013-05-23 17:55:45.175 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd710, ret 0, client 0x2327080 member 0x2327c30 [ CSSD]2013-05-23 17:55:45.175 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 974 [ CSSD]2013-05-23 17:55:45.175 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327c30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:55:45.377 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327120 [ CSSD]2013-05-23 17:55:45.377 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327120) con(0x22e35d0) skgpid ospid 15188 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:55:45.427 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:45.427 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:45.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:46.113 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:46.113 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:55:47.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:47.432 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:48.004 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:48.539 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:55:48.539 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 975 (0x7f431bfdd790) [ CSSD]2013-05-23 17:55:48.540 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd790 (RPC#975) tag(3cf002d) sent to node 1 [ CSSD]2013-05-23 17:55:48.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:48.540 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 975 (0x7f431bfdd790) [ CSSD]2013-05-23 17:55:48.540 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd790 (RPC#975) state 6, flags 0x202 [ CSSD]2013-05-23 17:55:48.540 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd790, ret 0, client (nil) [ CSSD]2013-05-23 17:55:48.540 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 975 [ CSSD]2013-05-23 17:55:49.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:49.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:50.033 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:51.116 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:51.116 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:51.489 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:51.489 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:52.038 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:53.514 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:53.514 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:54.041 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:55.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:55.199 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015562, gin 1, member 0 [ CSSD]2013-05-23 17:55:55.199 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:55:55.199 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:55:55.206 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:55.206 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:55:55.206 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:55:55.527 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:55.527 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:56.066 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:56.119 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:55:56.119 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:55:57.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:57.532 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:55:58.071 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:55:58.563 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:55:58.563 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 976 (0x7f431bfdd810) [ CSSD]2013-05-23 17:55:58.563 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd810 (RPC#976) tag(3d0002d) sent to node 1 [ CSSD]2013-05-23 17:55:58.564 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:58.564 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 976 (0x7f431bfdd810) [ CSSD]2013-05-23 17:55:58.564 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd810 (RPC#976) state 6, flags 0x202 [ CSSD]2013-05-23 17:55:58.564 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd810, ret 0, client (nil) [ CSSD]2013-05-23 17:55:58.564 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 976 [ CSSD]2013-05-23 17:55:59.538 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:55:59.538 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:00.106 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:00.415 [1191196992] >TRACE: clssgmAllocProc: (0x2328140) allocated [ CSSD]2013-05-23 17:56:00.416 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:56:00.417 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2328140 - 1,2,3 [ CSSD]2013-05-23 17:56:00.417 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9e90) proc(0x2328140) pid(15365) version 11:1:1:4 [ CSSD]2013-05-23 17:56:00.423 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2328140), client(1/0x23280a0) [ CSSD]2013-05-23 17:56:00.423 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:00.423 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23280a0 with con 0x2328410, requested num -1 [ CSSD]2013-05-23 17:56:00.423 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:00.423 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 977 (0x7f431bfdd890) [ CSSD]2013-05-23 17:56:00.423 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd890 (RPC#977) tag(3d1002d) sent to node 1 [ CSSD]2013-05-23 17:56:00.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d1002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:00.423 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015563, gin 1, member 0 [ CSSD]2013-05-23 17:56:00.423 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328a00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:56:00.424 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:56:00.424 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:56:00.424 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd890 (RPC#977) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:00.424 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdd890, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:00.424 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23280a0 [ CSSD]2013-05-23 17:56:00.424 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 977 [ CSSD]2013-05-23 17:56:00.431 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:00.431 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23280a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:00.431 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2328d00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:00.431 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 978 (0x7f431bfdd910) [ CSSD]2013-05-23 17:56:00.431 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd910 (RPC#978) tag(3d2002d) sent to node 1 [ CSSD]2013-05-23 17:56:00.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03d2002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:00.432 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:00.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd910 (RPC#978) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:00.432 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdd910, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:00.432 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 978 [ CSSD]2013-05-23 17:56:00.432 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2328d00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:56:00.634 [1191196992] >TRACE: clssgmDeadProc: proc 0x2328140 [ CSSD]2013-05-23 17:56:00.634 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2328140) con(0x22e9e90) skgpid ospid 15365 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:01.122 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:01.122 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:01.590 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:01.590 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:02.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:03.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:03.617 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:04.203 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:05.125 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:05.125 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:56:05.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:05.628 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:06.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:07.634 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:07.634 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:08.234 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:08.587 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:56:08.587 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 979 (0x7f431bfdd990) [ CSSD]2013-05-23 17:56:08.587 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdd990 (RPC#979) tag(3d3002d) sent to node 1 [ CSSD]2013-05-23 17:56:08.588 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:08.588 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 979 (0x7f431bfdd990) [ CSSD]2013-05-23 17:56:08.588 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdd990 (RPC#979) state 6, flags 0x202 [ CSSD]2013-05-23 17:56:08.588 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdd990, ret 0, client (nil) [ CSSD]2013-05-23 17:56:08.588 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 979 [ CSSD]2013-05-23 17:56:09.658 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:09.658 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:10.128 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:10.128 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:10.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:10.461 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:10.461 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015564, gin 1, member 0 [ CSSD]2013-05-23 17:56:10.461 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:56:10.461 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:56:10.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:10.469 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:56:10.469 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c0e6d60) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:56:11.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:11.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:12.351 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:13.716 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:13.716 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:14.424 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:15.131 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:15.131 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:15.666 [1191196992] >TRACE: clssgmAllocProc: (0x2328140) allocated [ CSSD]2013-05-23 17:56:15.667 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:56:15.667 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2328140 - 1,2,3 [ CSSD]2013-05-23 17:56:15.667 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9e90) proc(0x2328140) pid(15515) version 11:1:1:4 [ CSSD]2013-05-23 17:56:15.672 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2328140), client(1/0x23280a0) [ CSSD]2013-05-23 17:56:15.672 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:15.672 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23280a0 with con 0x2328410, requested num -1 [ CSSD]2013-05-23 17:56:15.672 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:15.672 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 980 (0x7f431bfdda10) [ CSSD]2013-05-23 17:56:15.673 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdda10 (RPC#980) tag(3d4002d) sent to node 1 [ CSSD]2013-05-23 17:56:15.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d4002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:15.673 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015565, gin 1, member 0 [ CSSD]2013-05-23 17:56:15.673 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328a00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:56:15.674 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:56:15.674 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:56:15.674 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdda10 (RPC#980) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:15.674 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdda10, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:15.674 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23280a0 [ CSSD]2013-05-23 17:56:15.674 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 980 [ CSSD]2013-05-23 17:56:15.680 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:15.680 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23280a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:15.680 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2328d00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:15.680 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 981 (0x7f431bfdda90) [ CSSD]2013-05-23 17:56:15.680 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdda90 (RPC#981) tag(3d5002d) sent to node 1 [ CSSD]2013-05-23 17:56:15.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03d5002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:15.680 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:15.680 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdda90 (RPC#981) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:15.680 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdda90, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:15.681 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 981 [ CSSD]2013-05-23 17:56:15.681 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2328d00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:56:15.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:15.728 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:15.883 [1191196992] >TRACE: clssgmDeadProc: proc 0x2328140 [ CSSD]2013-05-23 17:56:15.883 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2328140) con(0x22e9e90) skgpid ospid 15515 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:16.449 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:17.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:17.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:18.522 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:18.614 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:56:18.614 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 982 (0x7f431bfddb10) [ CSSD]2013-05-23 17:56:18.615 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddb10 (RPC#982) tag(3d6002d) sent to node 1 [ CSSD]2013-05-23 17:56:18.615 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:18.615 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 982 (0x7f431bfddb10) [ CSSD]2013-05-23 17:56:18.615 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddb10 (RPC#982) state 6, flags 0x202 [ CSSD]2013-05-23 17:56:18.615 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfddb10, ret 0, client (nil) [ CSSD]2013-05-23 17:56:18.615 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 982 [ CSSD]2013-05-23 17:56:19.760 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:19.760 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:20.134 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:20.134 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:20.567 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:21.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:21.810 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:22.573 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:23.064 [1191196992] >TRACE: clssgmAllocProc: (0x2328140) allocated [ CSSD]2013-05-23 17:56:23.065 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:56:23.066 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2328140 - 1,2,3 [ CSSD]2013-05-23 17:56:23.066 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9e90) proc(0x2328140) pid(15609) version 11:1:1:4 [ CSSD]2013-05-23 17:56:23.072 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2328140), client(1/0x23280a0) [ CSSD]2013-05-23 17:56:23.072 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:23.072 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x23280a0 with con 0x2328410, requested num -1 [ CSSD]2013-05-23 17:56:23.072 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:56:23.072 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 983 (0x7f431bfddb90) [ CSSD]2013-05-23 17:56:23.072 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddb90 (RPC#983) tag(3d7002d) sent to node 1 [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03d7002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015566, gin 1, member 0 [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328a00/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddb90 (RPC#983) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfddb90, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23280a0 [ CSSD]2013-05-23 17:56:23.072 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 983 [ CSSD]2013-05-23 17:56:23.078 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:23.078 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23280a0), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:56:23.078 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2328d00) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 17:56:23.078 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 984 (0x7f431bfddc10) [ CSSD]2013-05-23 17:56:23.078 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddc10 (RPC#984) tag(3d8002d) sent to node 1 [ CSSD]2013-05-23 17:56:23.078 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03d8002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:23.078 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 17:56:23.078 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddc10 (RPC#984) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:23.078 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfddc10, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:23.079 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 984 [ CSSD]2013-05-23 17:56:23.079 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x2328d00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:56:23.191 [1191196992] >TRACE: clssgmAllocProc: (0x2310cd0) allocated [ CSSD]2013-05-23 17:56:23.192 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:56:23.192 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2310cd0 - 1,2,3 [ CSSD]2013-05-23 17:56:23.192 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e35d0) proc(0x2310cd0) pid(15614) version 11:1:1:4 [ CSSD]2013-05-23 17:56:23.207 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:56:23.209 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:56:23.211 [1191196992] >TRACE: clssgmDeadProc: proc 0x2310cd0 [ CSSD]2013-05-23 17:56:23.211 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2310cd0) con(0x22e35d0) skgpid ospid 15614 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:23.281 [1191196992] >TRACE: clssgmDeadProc: proc 0x2328140 [ CSSD]2013-05-23 17:56:23.281 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2328140) con(0x22e9e90) skgpid ospid 15609 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:23.816 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:23.816 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:24.137 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:24.137 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:56:24.605 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:25.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:25.724 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015567, gin 1, member 0 [ CSSD]2013-05-23 17:56:25.724 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:56:25.724 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:56:25.731 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:25.731 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:56:25.731 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c109eb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:56:25.829 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:25.829 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:26.611 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:27.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:27.835 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:28.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:28.639 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:56:28.639 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 985 (0x7f431bfddc90) [ CSSD]2013-05-23 17:56:28.639 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddc90 (RPC#985) tag(3d9002d) sent to node 1 [ CSSD]2013-05-23 17:56:28.640 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:28.640 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 985 (0x7f431bfddc90) [ CSSD]2013-05-23 17:56:28.640 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddc90 (RPC#985) state 6, flags 0x202 [ CSSD]2013-05-23 17:56:28.640 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfddc90, ret 0, client (nil) [ CSSD]2013-05-23 17:56:28.640 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 985 [ CSSD]2013-05-23 17:56:29.140 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:29.140 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:29.886 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:29.886 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:30.627 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:30.921 [1191196992] >TRACE: clssgmAllocProc: (0x2328140) allocated [ CSSD]2013-05-23 17:56:30.922 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:56:30.923 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2328140 - 1,2,3 [ CSSD]2013-05-23 17:56:30.923 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e9e90) proc(0x2328140) pid(15697) version 11:1:1:4 [ CSSD]2013-05-23 17:56:30.928 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x2328140), client(1/0x23280a0) [ CSSD]2013-05-23 17:56:30.928 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:30.928 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23280a0 with con 0x2328410, requested num -1 [ CSSD]2013-05-23 17:56:30.928 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:30.928 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 986 (0x7f431bfddd10) [ CSSD]2013-05-23 17:56:30.928 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddd10 (RPC#986) tag(3da002d) sent to node 1 [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03da002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015568, gin 1, member 0 [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328a00/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddd10 (RPC#986) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfddd10, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23280a0 [ CSSD]2013-05-23 17:56:30.929 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 986 [ CSSD]2013-05-23 17:56:30.934 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23280a0) [ CSSD]2013-05-23 17:56:30.935 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23280a0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:30.935 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2328d00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:30.935 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 987 (0x7f431bfddd90) [ CSSD]2013-05-23 17:56:30.935 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddd90 (RPC#987) tag(3db002d) sent to node 1 [ CSSD]2013-05-23 17:56:30.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03db002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:30.935 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:30.935 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddd90 (RPC#987) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:30.935 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfddd90, ret 0, client 0x23280a0 member 0x2328d00 [ CSSD]2013-05-23 17:56:30.935 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 987 [ CSSD]2013-05-23 17:56:30.935 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2328d00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:56:31.138 [1191196992] >TRACE: clssgmDeadProc: proc 0x2328140 [ CSSD]2013-05-23 17:56:31.138 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2328140) con(0x22e9e90) skgpid ospid 15697 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:31.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:31.913 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:32.653 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:33.938 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:33.938 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:34.143 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:34.143 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:34.705 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:35.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:35.939 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:36.711 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:37.963 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:37.963 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:38.665 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:56:38.665 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 988 (0x7f431bfdde10) [ CSSD]2013-05-23 17:56:38.665 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdde10 (RPC#988) tag(3dc002d) sent to node 1 [ CSSD]2013-05-23 17:56:38.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:38.666 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 988 (0x7f431bfdde10) [ CSSD]2013-05-23 17:56:38.666 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdde10 (RPC#988) state 6, flags 0x202 [ CSSD]2013-05-23 17:56:38.666 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdde10, ret 0, client (nil) [ CSSD]2013-05-23 17:56:38.666 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 988 [ CSSD]2013-05-23 17:56:38.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:39.147 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:39.147 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:39.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:39.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:40.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:40.346 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015569, gin 1, member 0 [ CSSD]2013-05-23 17:56:40.346 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:56:40.346 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:56:40.354 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:40.354 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:56:40.354 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c10a490) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:56:40.729 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:40.984 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:40.984 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015570, gin 1, member 0 [ CSSD]2013-05-23 17:56:40.985 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:56:40.985 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:56:40.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:40.991 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:56:40.991 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c135050) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:56:42.002 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:42.002 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:42.754 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:44.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:44.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:44.149 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:44.150 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:44.218 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:56:44.219 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:56:44.219 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:56:44.219 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5900) proc(0x7f431c10a190) pid(15833) version 11:1:1:4 [ CSSD]2013-05-23 17:56:44.236 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c10a190), client(1/0x7f431c108e70) [ CSSD]2013-05-23 17:56:44.236 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:56:44.236 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:56:44.236 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:44.236 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 989 (0x7f431bfdde90) [ CSSD]2013-05-23 17:56:44.236 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdde90 (RPC#989) tag(3dd002d) sent to node 1 [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03dd002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015571, gin 1, member 0 [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdde90 (RPC#989) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdde90, ret 0, client 0x7f431c108e70 member 0x7f431c13c230 [ CSSD]2013-05-23 17:56:44.236 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 17:56:44.237 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 989 [ CSSD]2013-05-23 17:56:44.242 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:56:44.242 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:44.242 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:44.242 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 990 (0x7f431bfddf10) [ CSSD]2013-05-23 17:56:44.242 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddf10 (RPC#990) tag(3de002d) sent to node 1 [ CSSD]2013-05-23 17:56:44.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03de002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:44.243 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:44.243 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddf10 (RPC#990) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:44.243 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfddf10, ret 0, client 0x7f431c108e70 member 0x7f431c13c230 [ CSSD]2013-05-23 17:56:44.243 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 990 [ CSSD]2013-05-23 17:56:44.243 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:56:44.445 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:56:44.445 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c0e5900) skgpid ospid 15833 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:44.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:46.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:46.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:46.177 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:56:46.178 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:56:46.179 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:56:46.179 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c108e60) proc(0x7f431c11fd30) pid(15870) version 11:1:1:4 [ CSSD]2013-05-23 17:56:46.194 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c11fd30), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:56:46.194 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:56:46.194 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:56:46.195 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:46.195 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 991 (0x7f431bfddf90) [ CSSD]2013-05-23 17:56:46.195 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfddf90 (RPC#991) tag(3df002d) sent to node 1 [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03df002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015572, gin 1, member 0 [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfddf90 (RPC#991) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfddf90, ret 0, client 0x7f431c135120 member 0x7f431c13c230 [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:56:46.195 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 991 [ CSSD]2013-05-23 17:56:46.201 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:56:46.201 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:46.201 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:56:46.201 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 992 (0x7f431bfde010) [ CSSD]2013-05-23 17:56:46.201 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde010 (RPC#992) tag(3e0002d) sent to node 1 [ CSSD]2013-05-23 17:56:46.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e0002d) incarnation(174631051) [ CSSD]2013-05-23 17:56:46.202 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:56:46.202 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde010 (RPC#992) state 6, flags 0x100 [ CSSD]2013-05-23 17:56:46.202 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde010, ret 0, client 0x7f431c135120 member 0x7f431c13c230 [ CSSD]2013-05-23 17:56:46.202 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 992 [ CSSD]2013-05-23 17:56:46.202 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:56:46.404 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:56:46.404 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c108e60) skgpid ospid 15870 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:56:46.831 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:48.044 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:48.044 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:48.152 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:48.152 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:56:48.689 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:56:48.689 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 993 (0x7f431bfde090) [ CSSD]2013-05-23 17:56:48.689 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde090 (RPC#993) tag(3e1002d) sent to node 1 [ CSSD]2013-05-23 17:56:48.690 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:48.690 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 993 (0x7f431bfde090) [ CSSD]2013-05-23 17:56:48.690 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde090 (RPC#993) state 6, flags 0x202 [ CSSD]2013-05-23 17:56:48.690 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde090, ret 0, client (nil) [ CSSD]2013-05-23 17:56:48.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 993 [ CSSD]2013-05-23 17:56:48.843 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:50.098 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:50.098 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:50.848 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:52.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:52.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:52.854 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:53.155 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:53.155 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:54.148 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:54.148 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:54.907 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:56.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:56.200 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:56.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:56.236 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015573, gin 1, member 0 [ CSSD]2013-05-23 17:56:56.236 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:56:56.236 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:56:56.242 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:56.242 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:56:56.242 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:56:56.931 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:56:58.158 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:56:58.158 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:56:58.204 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:58.205 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:56:58.718 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:56:58.719 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 994 (0x7f431bfde110) [ CSSD]2013-05-23 17:56:58.719 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde110 (RPC#994) tag(3e2002d) sent to node 1 [ CSSD]2013-05-23 17:56:58.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:56:58.719 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 994 (0x7f431bfde110) [ CSSD]2013-05-23 17:56:58.719 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde110 (RPC#994) state 6, flags 0x202 [ CSSD]2013-05-23 17:56:58.719 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde110, ret 0, client (nil) [ CSSD]2013-05-23 17:56:58.720 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 994 [ CSSD]2013-05-23 17:56:58.944 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:00.217 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:00.217 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:00.950 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:01.435 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:57:01.436 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:57:01.436 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:57:01.436 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c10a190) pid(16013) version 11:1:1:4 [ CSSD]2013-05-23 17:57:01.442 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c10a190), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 17:57:01.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 17:57:01.442 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0cf710, requested num -1 [ CSSD]2013-05-23 17:57:01.442 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:01.442 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 995 (0x7f431bfde190) [ CSSD]2013-05-23 17:57:01.442 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde190 (RPC#995) tag(3e3002d) sent to node 1 [ CSSD]2013-05-23 17:57:01.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e3002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:01.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015574, gin 1, member 0 [ CSSD]2013-05-23 17:57:01.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:57:01.442 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:57:01.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:57:01.443 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde190 (RPC#995) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:01.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde190, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:01.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 17:57:01.443 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 995 [ CSSD]2013-05-23 17:57:01.448 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 17:57:01.448 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:01.448 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:01.448 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 996 (0x7f431bfde210) [ CSSD]2013-05-23 17:57:01.448 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde210 (RPC#996) tag(3e4002d) sent to node 1 [ CSSD]2013-05-23 17:57:01.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e4002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:01.449 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:01.449 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde210 (RPC#996) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:01.449 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde210, ret 0, client 0x7f431c0e6e30 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:01.449 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 996 [ CSSD]2013-05-23 17:57:01.449 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:57:01.651 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:57:01.651 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c12f0c0) skgpid ospid 16013 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:02.241 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:02.241 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:02.956 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:03.160 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:03.160 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:04.246 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:04.246 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:04.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:06.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:06.301 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:06.982 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:08.163 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:08.163 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:08.306 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:08.306 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:08.743 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:57:08.743 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 997 (0x7f431bfde290) [ CSSD]2013-05-23 17:57:08.743 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde290 (RPC#997) tag(3e5002d) sent to node 1 [ CSSD]2013-05-23 17:57:08.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:08.744 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 997 (0x7f431bfde290) [ CSSD]2013-05-23 17:57:08.744 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde290 (RPC#997) state 6, flags 0x202 [ CSSD]2013-05-23 17:57:08.744 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde290, ret 0, client (nil) [ CSSD]2013-05-23 17:57:08.744 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 997 [ CSSD]2013-05-23 17:57:08.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:10.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:10.318 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:11.001 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:11.486 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:11.486 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015575, gin 1, member 0 [ CSSD]2013-05-23 17:57:11.486 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:11.486 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:11.493 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:11.493 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:11.493 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:12.166 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:12.166 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:57:12.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:12.343 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:13.006 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:14.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:14.349 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:15.027 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:15.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:15.316 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015576, gin 1, member 0 [ CSSD]2013-05-23 17:57:15.317 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:15.317 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:15.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:15.324 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:15.324 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:16.421 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:16.421 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:16.682 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:57:16.683 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:57:16.683 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:57:16.683 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(16357) version 11:1:1:4 [ CSSD]2013-05-23 17:57:16.689 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c11fd30), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:57:16.689 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:57:16.689 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:57:16.689 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:16.689 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 998 (0x7f431bfde310) [ CSSD]2013-05-23 17:57:16.689 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde310 (RPC#998) tag(3e6002d) sent to node 1 [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e6002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015577, gin 1, member 0 [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde310 (RPC#998) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde310, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:57:16.689 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:57:16.690 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 998 [ CSSD]2013-05-23 17:57:16.694 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:57:16.694 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:16.694 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:16.694 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 999 (0x7f431bfde390) [ CSSD]2013-05-23 17:57:16.695 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde390 (RPC#999) tag(3e7002d) sent to node 1 [ CSSD]2013-05-23 17:57:16.695 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03e7002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:16.695 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:16.695 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde390 (RPC#999) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:16.695 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde390, ret 0, client 0x7f431c0e5910 member 0x7f431c12f520 [ CSSD]2013-05-23 17:57:16.695 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 999 [ CSSD]2013-05-23 17:57:16.695 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:57:16.897 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:57:16.897 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 16357 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:17.032 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:17.169 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:17.169 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:18.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:18.426 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:18.767 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:57:18.767 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1000 (0x7f431bfde410) [ CSSD]2013-05-23 17:57:18.767 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde410 (RPC#1000) tag(3e8002d) sent to node 1 [ CSSD]2013-05-23 17:57:18.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:18.768 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1000 (0x7f431bfde410) [ CSSD]2013-05-23 17:57:18.768 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde410 (RPC#1000) state 6, flags 0x202 [ CSSD]2013-05-23 17:57:18.768 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde410, ret 0, client (nil) [ CSSD]2013-05-23 17:57:18.768 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1000 [ CSSD]2013-05-23 17:57:19.044 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:20.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:20.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:21.069 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:21.797 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:57:21.798 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:57:21.799 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:57:21.799 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c10a190) pid(16433) version 11:1:1:4 [ CSSD]2013-05-23 17:57:21.920 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0955a0) allocated [ CSSD]2013-05-23 17:57:21.921 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:57:21.922 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0955a0 - 1,2,3 [ CSSD]2013-05-23 17:57:21.922 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c0955a0) pid(16443) version 11:1:1:4 [ CSSD]2013-05-23 17:57:21.937 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:57:21.939 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:57:21.941 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0955a0 [ CSSD]2013-05-23 17:57:21.941 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0955a0) con(0x7f431c12a550) skgpid ospid 16443 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:22.022 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(86/0x7f431c0e5910) [ CSSD]2013-05-23 17:57:22.022 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 86 (0x7f431c0e5910) [ CSSD]2013-05-23 17:57:22.022 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c0e5910 with con 0x7f431c10a550, requested num -1 [ CSSD]2013-05-23 17:57:22.022 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:57:22.022 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1001 (0x7f431bfde490) [ CSSD]2013-05-23 17:57:22.022 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde490 (RPC#1001) tag(3e9002d) sent to node 1 [ CSSD]2013-05-23 17:57:22.022 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03e9002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:22.022 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015578, gin 1, member 0 [ CSSD]2013-05-23 17:57:22.022 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c13edd0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 17:57:22.023 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 17:57:22.023 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 17:57:22.023 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde490 (RPC#1001) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:22.023 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde490, ret 0, client 0x7f431c0e5910 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:22.023 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:57:22.023 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1001 [ CSSD]2013-05-23 17:57:22.028 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 86 (0x7f431c0e5910) [ CSSD]2013-05-23 17:57:22.028 [1191196992] >TRACE: clssgmExitGrock: client 86 (0x7f431c0e5910), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:57:22.028 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 17:57:22.028 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1002 (0x7f431bfde510) [ CSSD]2013-05-23 17:57:22.028 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde510 (RPC#1002) tag(3ea002d) sent to node 1 [ CSSD]2013-05-23 17:57:22.029 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ea002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:22.029 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 17:57:22.029 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde510 (RPC#1002) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:22.029 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde510, ret 0, client 0x7f431c0e5910 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:22.029 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1002 [ CSSD]2013-05-23 17:57:22.029 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:57:22.030 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:57:22.030 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c12f0c0) skgpid ospid 16433 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:22.172 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:22.172 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:22.444 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:22.444 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:23.075 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:24.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:24.470 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:25.128 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:26.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:26.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:26.738 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:26.738 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015579, gin 1, member 0 [ CSSD]2013-05-23 17:57:26.738 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:26.738 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:26.745 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:26.745 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:26.745 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:27.134 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:27.174 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:27.174 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:28.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:28.528 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:28.790 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:57:28.790 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1003 (0x7f431bfde590) [ CSSD]2013-05-23 17:57:28.790 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde590 (RPC#1003) tag(3eb002d) sent to node 1 [ CSSD]2013-05-23 17:57:28.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:28.791 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1003 (0x7f431bfde590) [ CSSD]2013-05-23 17:57:28.791 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde590 (RPC#1003) state 6, flags 0x202 [ CSSD]2013-05-23 17:57:28.791 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde590, ret 0, client (nil) [ CSSD]2013-05-23 17:57:28.791 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1003 [ CSSD]2013-05-23 17:57:29.145 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:30.540 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:30.540 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:31.151 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:31.932 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:57:31.933 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:57:31.934 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:57:31.934 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c11fd30) pid(16516) version 11:1:1:4 [ CSSD]2013-05-23 17:57:31.939 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:57:31.939 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:57:31.939 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135f10, requested num -1 [ CSSD]2013-05-23 17:57:31.939 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:31.939 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1004 (0x7f431bfde610) [ CSSD]2013-05-23 17:57:31.939 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde610 (RPC#1004) tag(3ec002d) sent to node 1 [ CSSD]2013-05-23 17:57:31.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ec002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:31.939 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015580, gin 1, member 0 [ CSSD]2013-05-23 17:57:31.939 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:57:31.940 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:57:31.940 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:57:31.940 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde610 (RPC#1004) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:31.940 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde610, ret 0, client 0x7f431c0cf720 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:31.940 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:57:31.940 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1004 [ CSSD]2013-05-23 17:57:31.945 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:57:31.945 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:31.945 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:31.945 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1005 (0x7f431bfde690) [ CSSD]2013-05-23 17:57:31.945 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde690 (RPC#1005) tag(3ed002d) sent to node 1 [ CSSD]2013-05-23 17:57:31.945 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03ed002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:31.945 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:31.946 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde690 (RPC#1005) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:31.946 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde690, ret 0, client 0x7f431c0cf720 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:31.946 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1005 [ CSSD]2013-05-23 17:57:31.946 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:57:32.148 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:57:32.148 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12f0c0) skgpid ospid 16516 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:32.177 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:32.177 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:32.545 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:32.545 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:33.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:34.551 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:34.551 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:35.168 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:36.180 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:36.180 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:57:36.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:36.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:37.174 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:38.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:38.607 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:38.814 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:57:38.815 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1006 (0x7f431bfde710) [ CSSD]2013-05-23 17:57:38.815 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde710 (RPC#1006) tag(3ee002d) sent to node 1 [ CSSD]2013-05-23 17:57:38.815 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:38.815 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1006 (0x7f431bfde710) [ CSSD]2013-05-23 17:57:38.815 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde710 (RPC#1006) state 6, flags 0x202 [ CSSD]2013-05-23 17:57:38.815 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde710, ret 0, client (nil) [ CSSD]2013-05-23 17:57:38.816 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1006 [ CSSD]2013-05-23 17:57:39.226 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:40.610 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:40.610 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015581, gin 1, member 0 [ CSSD]2013-05-23 17:57:40.610 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:40.610 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:40.617 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:40.617 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:40.617 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:40.620 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:40.620 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:41.183 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:41.183 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:41.231 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:41.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:41.999 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015582, gin 1, member 0 [ CSSD]2013-05-23 17:57:41.999 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:42.000 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:42.007 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:42.007 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:42.007 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:42.625 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:42.625 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:43.277 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:44.479 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c10a190) allocated [ CSSD]2013-05-23 17:57:44.480 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:57:44.480 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c10a190 - 1,2,3 [ CSSD]2013-05-23 17:57:44.481 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135f10) proc(0x7f431c10a190) pid(16711) version 11:1:1:4 [ CSSD]2013-05-23 17:57:44.486 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c10a190), client(1/0x7f431c108e70) [ CSSD]2013-05-23 17:57:44.486 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:57:44.486 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0e6e20, requested num -1 [ CSSD]2013-05-23 17:57:44.486 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:44.486 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1007 (0x7f431bfde790) [ CSSD]2013-05-23 17:57:44.487 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde790 (RPC#1007) tag(3ef002d) sent to node 1 [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ef002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015583, gin 1, member 0 [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde790 (RPC#1007) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde790, ret 0, client 0x7f431c108e70 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:44.487 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 17:57:44.488 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1007 [ CSSD]2013-05-23 17:57:44.493 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:57:44.493 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:44.493 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:44.493 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1008 (0x7f431bfde810) [ CSSD]2013-05-23 17:57:44.493 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde810 (RPC#1008) tag(3f0002d) sent to node 1 [ CSSD]2013-05-23 17:57:44.493 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f0002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:44.493 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:44.493 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde810 (RPC#1008) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:44.493 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde810, ret 0, client 0x7f431c108e70 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:44.493 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1008 [ CSSD]2013-05-23 17:57:44.493 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:57:44.650 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:44.650 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:44.696 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c10a190 [ CSSD]2013-05-23 17:57:44.696 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c10a190) con(0x7f431c135f10) skgpid ospid 16711 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:44.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:44.840 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015584, gin 1, member 0 [ CSSD]2013-05-23 17:57:44.840 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:44.840 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:44.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:44.849 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:44.849 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:45.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:46.186 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:46.186 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:46.723 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:46.723 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:47.187 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 17:57:47.187 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:57:47.188 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 17:57:47.188 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c11fd30) pid(16739) version 11:1:1:4 [ CSSD]2013-05-23 17:57:47.193 [1191196992] >TRACE: clssgmRegisterClient: proc(63/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:57:47.193 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:57:47.193 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:57:47.193 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:47.193 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1009 (0x7f431bfde890) [ CSSD]2013-05-23 17:57:47.193 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde890 (RPC#1009) tag(3f1002d) sent to node 1 [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03f1002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015585, gin 1, member 0 [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde890 (RPC#1009) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfde890, ret 0, client 0x7f431c0cf720 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:57:47.194 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1009 [ CSSD]2013-05-23 17:57:47.199 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:57:47.199 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:47.199 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:57:47.199 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1010 (0x7f431bfde910) [ CSSD]2013-05-23 17:57:47.200 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde910 (RPC#1010) tag(3f2002d) sent to node 1 [ CSSD]2013-05-23 17:57:47.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f2002d) incarnation(174631051) [ CSSD]2013-05-23 17:57:47.200 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:57:47.200 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde910 (RPC#1010) state 6, flags 0x100 [ CSSD]2013-05-23 17:57:47.200 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfde910, ret 0, client 0x7f431c0cf720 member 0x7f431c13c230 [ CSSD]2013-05-23 17:57:47.200 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1010 [ CSSD]2013-05-23 17:57:47.200 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:57:47.354 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:47.402 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 17:57:47.402 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12a550) skgpid ospid 16739 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:48.748 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:48.748 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:48.832 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:57:48.832 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1011 (0x7f431bfde990) [ CSSD]2013-05-23 17:57:48.832 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfde990 (RPC#1011) tag(3f3002d) sent to node 1 [ CSSD]2013-05-23 17:57:48.833 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:48.833 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1011 (0x7f431bfde990) [ CSSD]2013-05-23 17:57:48.833 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfde990 (RPC#1011) state 6, flags 0x202 [ CSSD]2013-05-23 17:57:48.833 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfde990, ret 0, client (nil) [ CSSD]2013-05-23 17:57:48.833 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1011 [ CSSD]2013-05-23 17:57:49.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:50.769 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x7f431c12e6d0) [ CSSD]2013-05-23 17:57:50.769 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 17:57:50.769 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x7f431c12e6d0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 17:57:50.769 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 17:57:50.778 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c11ab50) [ CSSD]2013-05-23 17:57:50.778 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c11ab50), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:57:50.778 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c11ab50), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:57:50.778 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 17:57:50.778 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0838d0 [ CSSD]2013-05-23 17:57:50.778 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0838d0) con(0x7f431c13fa20) skgpid 11414 ospid 11414 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:57:50.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:50.800 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:51.189 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:51.189 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:57:51.371 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:52.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:52.805 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:53.424 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:54.830 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:54.830 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:55.191 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:57:55.191 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:57:55.450 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:56.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:56.844 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:57.253 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:57.253 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015586, gin 1, member 0 [ CSSD]2013-05-23 17:57:57.253 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:57:57.253 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:57:57.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:57.260 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:57:57.260 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:57:57.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:57:58.852 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:57:58.852 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1012 (0x7f431bfdea10) [ CSSD]2013-05-23 17:57:58.852 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdea10 (RPC#1012) tag(3f4002d) sent to node 1 [ CSSD]2013-05-23 17:57:58.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:58.853 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1012 (0x7f431bfdea10) [ CSSD]2013-05-23 17:57:58.853 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdea10 (RPC#1012) state 6, flags 0x202 [ CSSD]2013-05-23 17:57:58.853 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdea10, ret 0, client (nil) [ CSSD]2013-05-23 17:57:58.853 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1012 [ CSSD]2013-05-23 17:57:58.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:57:58.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:57:59.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:00.194 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:00.194 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:00.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:00.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:01.473 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:02.434 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c109eb0) allocated [ CSSD]2013-05-23 17:58:02.435 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:58:02.436 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c109eb0 - 1,2,3 [ CSSD]2013-05-23 17:58:02.436 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c109eb0) pid(16886) version 11:1:1:4 [ CSSD]2013-05-23 17:58:02.442 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c109eb0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:58:02.442 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:58:02.442 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 17:58:02.442 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:02.442 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1013 (0x7f431bfdea90) [ CSSD]2013-05-23 17:58:02.442 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdea90 (RPC#1013) tag(3f5002d) sent to node 1 [ CSSD]2013-05-23 17:58:02.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03f5002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:02.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015587, gin 1, member 0 [ CSSD]2013-05-23 17:58:02.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:58:02.443 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:58:02.443 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:58:02.443 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdea90 (RPC#1013) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:02.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdea90, ret 0, client 0x7f431c0e5910 member 0x7f431c134600 [ CSSD]2013-05-23 17:58:02.443 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:58:02.443 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1013 [ CSSD]2013-05-23 17:58:02.448 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:58:02.448 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:02.448 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134600) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:02.448 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1014 (0x7f431bfdeb10) [ CSSD]2013-05-23 17:58:02.449 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdeb10 (RPC#1014) tag(3f6002d) sent to node 1 [ CSSD]2013-05-23 17:58:02.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f6002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:02.449 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:02.449 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdeb10 (RPC#1014) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:02.449 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdeb10, ret 0, client 0x7f431c0e5910 member 0x7f431c134600 [ CSSD]2013-05-23 17:58:02.449 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1014 [ CSSD]2013-05-23 17:58:02.449 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:58:02.652 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c109eb0 [ CSSD]2013-05-23 17:58:02.652 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c109eb0) con(0x7f431c12a550) skgpid ospid 16886 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:58:02.957 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:02.958 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:03.524 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:05.009 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:05.009 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:05.197 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:05.197 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:05.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:07.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:07.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:07.555 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:08.882 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:58:08.882 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1015 (0x7f431bfdeb90) [ CSSD]2013-05-23 17:58:08.882 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdeb90 (RPC#1015) tag(3f7002d) sent to node 1 [ CSSD]2013-05-23 17:58:08.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:08.883 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1015 (0x7f431bfdeb90) [ CSSD]2013-05-23 17:58:08.883 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdeb90 (RPC#1015) state 6, flags 0x202 [ CSSD]2013-05-23 17:58:08.883 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdeb90, ret 0, client (nil) [ CSSD]2013-05-23 17:58:08.883 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1015 [ CSSD]2013-05-23 17:58:09.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:09.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:09.606 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:10.200 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:10.200 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:11.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:11.050 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:11.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:12.504 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:12.504 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015588, gin 1, member 0 [ CSSD]2013-05-23 17:58:12.504 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:58:12.504 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:58:12.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:12.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:58:12.511 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:58:13.056 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:13.056 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:13.625 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:15.110 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:15.110 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:15.204 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:15.204 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:15.631 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:17.116 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:17.116 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:17.657 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:17.684 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e6d60) allocated [ CSSD]2013-05-23 17:58:17.684 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:58:17.685 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e6d60 - 1,2,3 [ CSSD]2013-05-23 17:58:17.685 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c0e6d60) pid(17038) version 11:1:1:4 [ CSSD]2013-05-23 17:58:17.690 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0e6d60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:58:17.690 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:58:17.690 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 17:58:17.690 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:17.690 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1016 (0x7f431bfdec10) [ CSSD]2013-05-23 17:58:17.690 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdec10 (RPC#1016) tag(3f8002d) sent to node 1 [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03f8002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015589, gin 1, member 0 [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdec10 (RPC#1016) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdec10, ret 0, client 0x7f431c135120 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:58:17.691 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1016 [ CSSD]2013-05-23 17:58:17.696 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:58:17.696 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:17.696 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:17.696 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1017 (0x7f431bfdec90) [ CSSD]2013-05-23 17:58:17.697 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdec90 (RPC#1017) tag(3f9002d) sent to node 1 [ CSSD]2013-05-23 17:58:17.697 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03f9002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:17.697 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:17.697 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdec90 (RPC#1017) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:17.697 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdec90, ret 0, client 0x7f431c135120 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:58:17.697 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1017 [ CSSD]2013-05-23 17:58:17.697 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:58:17.899 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e6d60 [ CSSD]2013-05-23 17:58:17.899 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e6d60) con(0x7f431c0840c0) skgpid ospid 17038 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:58:18.912 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:58:18.912 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1018 (0x7f431bfded10) [ CSSD]2013-05-23 17:58:18.912 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfded10 (RPC#1018) tag(3fa002d) sent to node 1 [ CSSD]2013-05-23 17:58:18.913 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:18.913 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1018 (0x7f431bfded10) [ CSSD]2013-05-23 17:58:18.913 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfded10 (RPC#1018) state 6, flags 0x202 [ CSSD]2013-05-23 17:58:18.913 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfded10, ret 0, client (nil) [ CSSD]2013-05-23 17:58:18.913 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1018 [ CSSD]2013-05-23 17:58:19.121 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:19.121 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:19.667 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:20.207 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:20.207 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:21.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:21.132 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:21.673 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:23.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:23.158 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:23.676 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:25.209 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:25.209 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:25.210 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:25.210 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:25.700 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:27.216 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:27.216 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:27.706 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:27.755 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:27.755 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015590, gin 1, member 0 [ CSSD]2013-05-23 17:58:27.755 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:58:27.755 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:58:27.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:27.762 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:58:27.762 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:58:28.943 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:58:28.943 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1019 (0x7f431bfded90) [ CSSD]2013-05-23 17:58:28.943 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfded90 (RPC#1019) tag(3fb002d) sent to node 1 [ CSSD]2013-05-23 17:58:28.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:28.944 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1019 (0x7f431bfded90) [ CSSD]2013-05-23 17:58:28.944 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfded90 (RPC#1019) state 6, flags 0x202 [ CSSD]2013-05-23 17:58:28.944 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfded90, ret 0, client (nil) [ CSSD]2013-05-23 17:58:28.944 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1019 [ CSSD]2013-05-23 17:58:29.221 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:29.221 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:29.738 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:30.212 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:30.212 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:31.233 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:31.233 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:31.743 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:32.930 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135050) allocated [ CSSD]2013-05-23 17:58:32.931 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:58:32.932 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135050 - 1,2,3 [ CSSD]2013-05-23 17:58:32.932 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c135050) pid(17205) version 11:1:1:4 [ CSSD]2013-05-23 17:58:32.937 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c135050), client(1/0x7f431c121830) [ CSSD]2013-05-23 17:58:32.937 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 17:58:32.937 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121830 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:58:32.937 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:32.937 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1020 (0x7f431bfdee10) [ CSSD]2013-05-23 17:58:32.937 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdee10 (RPC#1020) tag(3fc002d) sent to node 1 [ CSSD]2013-05-23 17:58:32.937 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03fc002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:32.937 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015591, gin 1, member 0 [ CSSD]2013-05-23 17:58:32.937 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:58:32.938 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:58:32.938 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:58:32.938 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdee10 (RPC#1020) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:32.938 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdee10, ret 0, client 0x7f431c121830 member 0x7f431c134600 [ CSSD]2013-05-23 17:58:32.938 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121830 [ CSSD]2013-05-23 17:58:32.938 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1020 [ CSSD]2013-05-23 17:58:32.944 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 17:58:32.944 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121830), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:32.944 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134600) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:32.944 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1021 (0x7f431bfdee90) [ CSSD]2013-05-23 17:58:32.944 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdee90 (RPC#1021) tag(3fd002d) sent to node 1 [ CSSD]2013-05-23 17:58:32.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(03fd002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:32.944 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:32.944 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdee90 (RPC#1021) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:32.944 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdee90, ret 0, client 0x7f431c121830 member 0x7f431c134600 [ CSSD]2013-05-23 17:58:32.944 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1021 [ CSSD]2013-05-23 17:58:32.944 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:58:33.147 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135050 [ CSSD]2013-05-23 17:58:33.147 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135050) con(0x7f431c0e5200) skgpid ospid 17205 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:58:33.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:33.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:33.836 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:35.215 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:35.216 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:35.311 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:35.311 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:35.842 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:37.316 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:37.316 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:37.848 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:38.973 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:58:38.974 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1022 (0x7f431bfdef10) [ CSSD]2013-05-23 17:58:38.974 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdef10 (RPC#1022) tag(3fe002d) sent to node 1 [ CSSD]2013-05-23 17:58:38.974 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:38.974 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1022 (0x7f431bfdef10) [ CSSD]2013-05-23 17:58:38.974 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdef10 (RPC#1022) state 6, flags 0x202 [ CSSD]2013-05-23 17:58:38.974 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdef10, ret 0, client (nil) [ CSSD]2013-05-23 17:58:38.974 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1022 [ CSSD]2013-05-23 17:58:39.218 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:39.218 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:58:39.321 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:39.321 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:39.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:40.870 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:40.870 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015592, gin 1, member 0 [ CSSD]2013-05-23 17:58:40.870 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:58:40.870 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:58:40.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:40.875 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:58:40.875 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:58:41.333 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:41.333 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:41.865 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:43.008 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:43.008 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015593, gin 1, member 0 [ CSSD]2013-05-23 17:58:43.008 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:58:43.008 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:58:43.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:43.015 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:58:43.015 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:58:43.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:43.339 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:43.877 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:44.221 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:44.222 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:44.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:44.359 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015594, gin 1, member 0 [ CSSD]2013-05-23 17:58:44.359 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 17:58:44.359 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:58:44.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:44.366 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 17:58:44.366 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22e9670) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:58:44.492 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:58:44.492 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:58:44.494 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:58:44.494 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:58:44.496 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:58:44.496 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:58:44.727 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0cf650) allocated [ CSSD]2013-05-23 17:58:44.728 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:58:44.729 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0cf650 - 1,2,3 [ CSSD]2013-05-23 17:58:44.729 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c0cf650) pid(17325) version 11:1:1:4 [ CSSD]2013-05-23 17:58:44.735 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0cf650), client(1/0x7f431c109f80) [ CSSD]2013-05-23 17:58:44.735 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:58:44.735 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c13fa20, requested num -1 [ CSSD]2013-05-23 17:58:44.735 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:44.735 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1023 (0x7f431bfdef90) [ CSSD]2013-05-23 17:58:44.735 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdef90 (RPC#1023) tag(3ff002d) sent to node 1 [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(03ff002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015595, gin 1, member 0 [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdef90 (RPC#1023) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdef90, ret 0, client 0x7f431c109f80 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 17:58:44.736 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1023 [ CSSD]2013-05-23 17:58:44.743 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 17:58:44.743 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:44.743 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c11fd30) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:44.743 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1024 (0x7f431bfdf010) [ CSSD]2013-05-23 17:58:44.743 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf010 (RPC#1024) tag(400002d) sent to node 1 [ CSSD]2013-05-23 17:58:44.743 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0400002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:44.743 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:44.743 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf010 (RPC#1024) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:44.743 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf010, ret 0, client 0x7f431c109f80 member 0x7f431c11fd30 [ CSSD]2013-05-23 17:58:44.743 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1024 [ CSSD]2013-05-23 17:58:44.743 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c11fd30) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:58:44.946 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0cf650 [ CSSD]2013-05-23 17:58:44.946 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0cf650) con(0x7f431c12f0c0) skgpid ospid 17325 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:58:45.411 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:45.411 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:45.882 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:47.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:47.417 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:47.906 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:48.187 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e6d60) allocated [ CSSD]2013-05-23 17:58:48.187 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:58:48.188 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e6d60 - 1,2,3 [ CSSD]2013-05-23 17:58:48.188 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c0e6d60) pid(17352) version 11:1:1:4 [ CSSD]2013-05-23 17:58:48.193 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0e6d60), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:58:48.193 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:58:48.193 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:58:48.193 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:48.193 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1025 (0x7f431bfdf090) [ CSSD]2013-05-23 17:58:48.193 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf090 (RPC#1025) tag(401002d) sent to node 1 [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0401002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015596, gin 1, member 0 [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf090 (RPC#1025) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf090, ret 0, client 0x7f431c135120 member 0x7f431c134600 [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:58:48.194 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1025 [ CSSD]2013-05-23 17:58:48.200 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:58:48.200 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:48.200 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134600) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:58:48.200 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1026 (0x7f431bfdf110) [ CSSD]2013-05-23 17:58:48.200 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf110 (RPC#1026) tag(402002d) sent to node 1 [ CSSD]2013-05-23 17:58:48.201 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0402002d) incarnation(174631051) [ CSSD]2013-05-23 17:58:48.201 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:58:48.201 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf110 (RPC#1026) state 6, flags 0x100 [ CSSD]2013-05-23 17:58:48.201 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf110, ret 0, client 0x7f431c135120 member 0x7f431c134600 [ CSSD]2013-05-23 17:58:48.201 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1026 [ CSSD]2013-05-23 17:58:48.201 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134600) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:58:48.403 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e6d60 [ CSSD]2013-05-23 17:58:48.403 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e6d60) con(0x7f431c0840c0) skgpid ospid 17352 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:58:48.993 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:58:48.994 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1027 (0x7f431bfdf190) [ CSSD]2013-05-23 17:58:48.994 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf190 (RPC#1027) tag(403002d) sent to node 1 [ CSSD]2013-05-23 17:58:48.994 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:48.994 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1027 (0x7f431bfdf190) [ CSSD]2013-05-23 17:58:48.994 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf190 (RPC#1027) state 6, flags 0x202 [ CSSD]2013-05-23 17:58:48.994 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf190, ret 0, client (nil) [ CSSD]2013-05-23 17:58:48.995 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1027 [ CSSD]2013-05-23 17:58:49.224 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:49.225 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:49.423 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:49.423 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:49.979 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:51.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:51.434 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:51.984 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:53.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:53.459 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:54.037 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:54.228 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:54.228 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:55.338 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22d7cd0) [ CSSD]2013-05-23 17:58:55.338 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22d7cd0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 17:58:55.338 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22d7cd0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 17:58:55.339 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 17:58:55.339 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1355f0 [ CSSD]2013-05-23 17:58:55.339 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1355f0) con(0x7f431c0e6720) skgpid 12082 ospid 12082 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:58:55.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:55.511 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:56.082 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:57.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:57.517 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:58:58.088 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:58:58.260 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:58.260 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015597, gin 1, member 0 [ CSSD]2013-05-23 17:58:58.260 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:58:58.260 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:58:58.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:58.266 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:58:58.267 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:58:59.012 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:58:59.013 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1028 (0x7f431bfdf210) [ CSSD]2013-05-23 17:58:59.013 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf210 (RPC#1028) tag(404002d) sent to node 1 [ CSSD]2013-05-23 17:58:59.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:59.013 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1028 (0x7f431bfdf210) [ CSSD]2013-05-23 17:58:59.013 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf210 (RPC#1028) state 6, flags 0x202 [ CSSD]2013-05-23 17:58:59.013 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf210, ret 0, client (nil) [ CSSD]2013-05-23 17:58:59.014 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1028 [ CSSD]2013-05-23 17:58:59.230 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:58:59.230 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:58:59.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:58:59.523 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:00.140 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:01.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:01.553 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:02.146 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:03.233 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:03.233 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:59:03.434 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c121760) allocated [ CSSD]2013-05-23 17:59:03.435 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:59:03.435 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c121760 - 1,2,3 [ CSSD]2013-05-23 17:59:03.435 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c121760) pid(17530) version 11:1:1:4 [ CSSD]2013-05-23 17:59:03.441 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c121760), client(1/0x7f431c108e70) [ CSSD]2013-05-23 17:59:03.441 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:59:03.441 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c108e70 with con 0x7f431c0840c0, requested num -1 [ CSSD]2013-05-23 17:59:03.441 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:03.441 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1029 (0x7f431bfdf290) [ CSSD]2013-05-23 17:59:03.441 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf290 (RPC#1029) tag(405002d) sent to node 1 [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0405002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015598, gin 1, member 0 [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf290 (RPC#1029) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf290, ret 0, client 0x7f431c108e70 member 0x7f431c0e6d60 [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c108e70 [ CSSD]2013-05-23 17:59:03.442 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1029 [ CSSD]2013-05-23 17:59:03.448 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c108e70) [ CSSD]2013-05-23 17:59:03.448 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c108e70), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:03.448 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:03.448 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1030 (0x7f431bfdf310) [ CSSD]2013-05-23 17:59:03.448 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf310 (RPC#1030) tag(406002d) sent to node 1 [ CSSD]2013-05-23 17:59:03.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0406002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:03.449 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:03.449 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf310 (RPC#1030) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:03.449 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf310, ret 0, client 0x7f431c108e70 member 0x7f431c0e6d60 [ CSSD]2013-05-23 17:59:03.449 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1030 [ CSSD]2013-05-23 17:59:03.449 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:59:03.560 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:03.560 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:03.651 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c121760 [ CSSD]2013-05-23 17:59:03.651 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c121760) con(0x7f431c11ab50) skgpid ospid 17530 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:59:04.157 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:05.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:05.612 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:06.162 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:07.618 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:07.618 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:08.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:08.235 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:08.236 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:09.033 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:59:09.033 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1031 (0x7f431bfdf390) [ CSSD]2013-05-23 17:59:09.033 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf390 (RPC#1031) tag(407002d) sent to node 1 [ CSSD]2013-05-23 17:59:09.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:09.034 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1031 (0x7f431bfdf390) [ CSSD]2013-05-23 17:59:09.034 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf390 (RPC#1031) state 6, flags 0x202 [ CSSD]2013-05-23 17:59:09.034 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf390, ret 0, client (nil) [ CSSD]2013-05-23 17:59:09.034 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1031 [ CSSD]2013-05-23 17:59:09.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:09.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:10.261 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:11.651 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:11.651 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:12.266 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:13.238 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:13.238 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:13.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:13.509 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015599, gin 1, member 0 [ CSSD]2013-05-23 17:59:13.509 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:59:13.509 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:59:13.515 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:13.515 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:59:13.515 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:59:13.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:13.656 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:14.279 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:15.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:15.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:16.285 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:17.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:17.686 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:18.241 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:18.241 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:18.309 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:18.682 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c135050) allocated [ CSSD]2013-05-23 17:59:18.683 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:59:18.684 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c135050 - 1,2,3 [ CSSD]2013-05-23 17:59:18.684 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c135050) pid(17674) version 11:1:1:4 [ CSSD]2013-05-23 17:59:18.700 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c135050), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 17:59:18.700 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:59:18.700 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 17:59:18.700 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:18.700 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1032 (0x7f431bfdf410) [ CSSD]2013-05-23 17:59:18.700 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf410 (RPC#1032) tag(408002d) sent to node 1 [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0408002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015600, gin 1, member 0 [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf410 (RPC#1032) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf410, ret 0, client 0x7f431c0cf720 member 0x7f431c108da0 [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 17:59:18.701 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1032 [ CSSD]2013-05-23 17:59:18.708 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 17:59:18.708 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:18.708 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108da0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:18.708 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1033 (0x7f431bfdf490) [ CSSD]2013-05-23 17:59:18.708 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf490 (RPC#1033) tag(409002d) sent to node 1 [ CSSD]2013-05-23 17:59:18.708 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0409002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:18.708 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:18.708 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf490 (RPC#1033) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:18.708 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf490, ret 0, client 0x7f431c0cf720 member 0x7f431c108da0 [ CSSD]2013-05-23 17:59:18.709 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1033 [ CSSD]2013-05-23 17:59:18.709 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c108da0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:59:18.911 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c135050 [ CSSD]2013-05-23 17:59:18.911 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c135050) con(0x7f431c0840c0) skgpid ospid 17674 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:59:19.063 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:59:19.063 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1034 (0x7f431bfdf510) [ CSSD]2013-05-23 17:59:19.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf510 (RPC#1034) tag(40a002d) sent to node 1 [ CSSD]2013-05-23 17:59:19.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:19.064 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1034 (0x7f431bfdf510) [ CSSD]2013-05-23 17:59:19.064 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf510 (RPC#1034) state 6, flags 0x202 [ CSSD]2013-05-23 17:59:19.064 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf510, ret 0, client (nil) [ CSSD]2013-05-23 17:59:19.064 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1034 [ CSSD]2013-05-23 17:59:19.692 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:19.692 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:20.360 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:21.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:21.724 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:22.366 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:23.244 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:23.244 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:23.729 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:23.729 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:24.378 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:25.742 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:25.742 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:26.383 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:27.246 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:27.247 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:59:27.766 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:27.766 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:28.436 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:28.770 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:28.770 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015601, gin 1, member 0 [ CSSD]2013-05-23 17:59:28.770 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:59:28.770 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:59:28.778 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:28.778 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:59:28.778 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:59:29.103 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:59:29.103 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1035 (0x7f431bfdf590) [ CSSD]2013-05-23 17:59:29.103 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf590 (RPC#1035) tag(40b002d) sent to node 1 [ CSSD]2013-05-23 17:59:29.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:29.104 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1035 (0x7f431bfdf590) [ CSSD]2013-05-23 17:59:29.104 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf590 (RPC#1035) state 6, flags 0x202 [ CSSD]2013-05-23 17:59:29.104 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf590, ret 0, client (nil) [ CSSD]2013-05-23 17:59:29.104 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1035 [ CSSD]2013-05-23 17:59:29.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:29.772 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:30.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:31.794 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:59:31.794 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:59:31.795 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:59:31.795 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 17:59:31.825 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:31.826 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:31.876 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:31.876 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015602, gin 1, member 0 [ CSSD]2013-05-23 17:59:31.876 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22e9900/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 17:59:31.876 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 17:59:31.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:31.883 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 17:59:31.883 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:59:32.250 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:32.250 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:32.467 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:33.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:33.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:33.954 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 17:59:33.955 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:59:33.956 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 17:59:33.956 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108da0) pid(17817) version 11:1:1:4 [ CSSD]2013-05-23 17:59:33.962 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c108da0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 17:59:33.962 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:59:33.962 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e5910 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:59:33.962 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:33.962 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1036 (0x7f431bfdf610) [ CSSD]2013-05-23 17:59:33.962 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf610 (RPC#1036) tag(40c002d) sent to node 1 [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(040c002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015603, gin 1, member 0 [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf610 (RPC#1036) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf610, ret 0, client 0x7f431c0e5910 member 0x7f431c135050 [ CSSD]2013-05-23 17:59:33.962 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e5910 [ CSSD]2013-05-23 17:59:33.963 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1036 [ CSSD]2013-05-23 17:59:33.969 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 17:59:33.969 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e5910), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:33.969 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135050) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:33.969 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1037 (0x7f431bfdf690) [ CSSD]2013-05-23 17:59:33.969 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf690 (RPC#1037) tag(40d002d) sent to node 1 [ CSSD]2013-05-23 17:59:33.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(040d002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:33.970 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:33.970 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf690 (RPC#1037) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:33.970 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf690, ret 0, client 0x7f431c0e5910 member 0x7f431c135050 [ CSSD]2013-05-23 17:59:33.970 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1037 [ CSSD]2013-05-23 17:59:33.970 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135050) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:59:34.173 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 17:59:34.173 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c0e6720) skgpid ospid 17817 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:59:34.560 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:35.844 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:35.844 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:36.566 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:37.253 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:37.253 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:37.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:37.849 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:38.577 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:39.124 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:59:39.125 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1038 (0x7f431bfdf710) [ CSSD]2013-05-23 17:59:39.125 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf710 (RPC#1038) tag(40e002d) sent to node 1 [ CSSD]2013-05-23 17:59:39.125 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:39.125 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1038 (0x7f431bfdf710) [ CSSD]2013-05-23 17:59:39.125 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf710 (RPC#1038) state 6, flags 0x202 [ CSSD]2013-05-23 17:59:39.125 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf710, ret 0, client (nil) [ CSSD]2013-05-23 17:59:39.126 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1038 [ CSSD]2013-05-23 17:59:39.921 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:39.921 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:40.583 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:41.125 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:41.125 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015604, gin 1, member 0 [ CSSD]2013-05-23 17:59:41.125 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:59:41.125 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:59:41.132 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:41.132 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:59:41.132 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:59:41.927 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:41.927 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:42.256 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:42.256 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:42.589 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:43.932 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:43.932 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:44.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:44.026 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015605, gin 1, member 0 [ CSSD]2013-05-23 17:59:44.026 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:59:44.026 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:59:44.032 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:44.032 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:59:44.033 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 17:59:44.609 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:44.984 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c109eb0) allocated [ CSSD]2013-05-23 17:59:44.985 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 17:59:44.986 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c109eb0 - 1,2,3 [ CSSD]2013-05-23 17:59:44.986 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c109eb0) pid(17934) version 11:1:1:4 [ CSSD]2013-05-23 17:59:44.991 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c109eb0), client(1/0x7f431c135120) [ CSSD]2013-05-23 17:59:44.991 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:59:44.991 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135120 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 17:59:44.991 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:44.991 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1039 (0x7f431bfdf790) [ CSSD]2013-05-23 17:59:44.991 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf790 (RPC#1039) tag(40f002d) sent to node 1 [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(040f002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015606, gin 1, member 0 [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf790 (RPC#1039) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf790, ret 0, client 0x7f431c135120 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135120 [ CSSD]2013-05-23 17:59:44.992 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1039 [ CSSD]2013-05-23 17:59:44.998 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135120) [ CSSD]2013-05-23 17:59:44.998 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135120), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:44.998 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:44.998 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1040 (0x7f431bfdf810) [ CSSD]2013-05-23 17:59:44.998 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf810 (RPC#1040) tag(410002d) sent to node 1 [ CSSD]2013-05-23 17:59:44.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0410002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:44.998 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:44.998 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf810 (RPC#1040) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:44.998 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf810, ret 0, client 0x7f431c135120 member 0x7f431c0e5840 [ CSSD]2013-05-23 17:59:44.998 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1040 [ CSSD]2013-05-23 17:59:44.998 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:59:45.201 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c109eb0 [ CSSD]2013-05-23 17:59:45.201 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c109eb0) con(0x7f431c0e6720) skgpid ospid 17934 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:59:45.944 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:45.944 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:46.259 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:46.259 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 17:59:46.614 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:47.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:47.949 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:48.647 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:49.164 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:59:49.164 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1041 (0x7f431bfdf890) [ CSSD]2013-05-23 17:59:49.164 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf890 (RPC#1041) tag(411002d) sent to node 1 [ CSSD]2013-05-23 17:59:49.165 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:49.165 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1041 (0x7f431bfdf890) [ CSSD]2013-05-23 17:59:49.165 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf890 (RPC#1041) state 6, flags 0x202 [ CSSD]2013-05-23 17:59:49.165 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdf890, ret 0, client (nil) [ CSSD]2013-05-23 17:59:49.165 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1041 [ CSSD]2013-05-23 17:59:49.212 [1191196992] >TRACE: clssgmAllocProc: (0x2327d10) allocated [ CSSD]2013-05-23 17:59:49.213 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 17:59:49.214 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327d10 - 1,2,3 [ CSSD]2013-05-23 17:59:49.214 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e35d0) proc(0x2327d10) pid(17984) version 11:1:1:4 [ CSSD]2013-05-23 17:59:49.219 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2327d10), client(1/0x2328400) [ CSSD]2013-05-23 17:59:49.219 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2328400) [ CSSD]2013-05-23 17:59:49.219 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2328400 with con 0x230f170, requested num -1 [ CSSD]2013-05-23 17:59:49.219 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:49.219 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1042 (0x7f431bfdf910) [ CSSD]2013-05-23 17:59:49.219 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf910 (RPC#1042) tag(412002d) sent to node 1 [ CSSD]2013-05-23 17:59:49.219 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0412002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:49.219 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015607, gin 1, member 0 [ CSSD]2013-05-23 17:59:49.219 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2328570/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 17:59:49.220 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 17:59:49.220 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 17:59:49.220 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf910 (RPC#1042) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:49.220 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdf910, ret 0, client 0x2328400 member 0x2327bf0 [ CSSD]2013-05-23 17:59:49.220 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2328400 [ CSSD]2013-05-23 17:59:49.220 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1042 [ CSSD]2013-05-23 17:59:49.225 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2328400) [ CSSD]2013-05-23 17:59:49.225 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2328400), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:49.225 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2327bf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 17:59:49.225 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1043 (0x7f431bfdf990) [ CSSD]2013-05-23 17:59:49.226 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdf990 (RPC#1043) tag(413002d) sent to node 1 [ CSSD]2013-05-23 17:59:49.226 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0413002d) incarnation(174631051) [ CSSD]2013-05-23 17:59:49.226 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 17:59:49.226 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdf990 (RPC#1043) state 6, flags 0x100 [ CSSD]2013-05-23 17:59:49.226 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdf990, ret 0, client 0x2328400 member 0x2327bf0 [ CSSD]2013-05-23 17:59:49.226 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1043 [ CSSD]2013-05-23 17:59:49.226 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2327bf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 17:59:49.428 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327d10 [ CSSD]2013-05-23 17:59:49.428 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327d10) con(0x22e35d0) skgpid ospid 17984 with 0 clients, refcount 0 [ CSSD]2013-05-23 17:59:50.021 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:50.021 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:50.652 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:51.262 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:51.262 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:52.027 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:52.027 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:52.658 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:54.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:54.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:54.671 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:56.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:56.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:56.265 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 17:59:56.265 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 17:59:56.677 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:58.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:58.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 17:59:58.688 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 17:59:59.194 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 17:59:59.194 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1044 (0x7f431bfdfa10) [ CSSD]2013-05-23 17:59:59.195 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfa10 (RPC#1044) tag(414002d) sent to node 1 [ CSSD]2013-05-23 17:59:59.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:59.195 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1044 (0x7f431bfdfa10) [ CSSD]2013-05-23 17:59:59.195 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfa10 (RPC#1044) state 6, flags 0x202 [ CSSD]2013-05-23 17:59:59.195 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfa10, ret 0, client (nil) [ CSSD]2013-05-23 17:59:59.195 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1044 [ CSSD]2013-05-23 17:59:59.288 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:59.288 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015608, gin 1, member 0 [ CSSD]2013-05-23 17:59:59.288 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22e9900/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 17:59:59.288 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 17:59:59.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 17:59:59.296 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 17:59:59.296 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x205ecb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:00:00.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:00.122 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:00.644 [1191196992] >TRACE: clssgmAllocProc: (0x2327d10) allocated [ CSSD]2013-05-23 18:00:00.645 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:00:00.646 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2327d10 - 1,2,3 [ CSSD]2013-05-23 18:00:00.646 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e35d0) proc(0x2327d10) pid(18099/18099) version 11:1:1:4 [ CSSD]2013-05-23 18:00:00.650 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2327d10), client(1/0x2328270) [ CSSD]2013-05-23 18:00:00.650 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2328270) [ CSSD]2013-05-23 18:00:00.650 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 18:00:00.650 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 18:00:00.652 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2327d10), client(2/0x2328770) [ CSSD]2013-05-23 18:00:00.652 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2328770) [ CSSD]2013-05-23 18:00:00.652 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:00:00.652 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 18:00:00.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:01.268 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:01.268 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:02.127 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:02.127 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:02.700 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:04.153 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:04.153 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:04.459 [1191196992] >TRACE: clssgmAllocProc: (0x23112c0) allocated [ CSSD]2013-05-23 18:00:04.460 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:00:04.461 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x23112c0 - 1,2,3 [ CSSD]2013-05-23 18:00:04.461 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2328e50) proc(0x23112c0) pid(18306) version 11:1:1:4 [ CSSD]2013-05-23 18:00:04.467 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x23112c0), client(1/0x23119b0) [ CSSD]2013-05-23 18:00:04.467 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x23119b0) [ CSSD]2013-05-23 18:00:04.467 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x23119b0 with con 0x23292f0, requested num -1 [ CSSD]2013-05-23 18:00:04.467 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:04.467 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1045 (0x7f431bfdfa90) [ CSSD]2013-05-23 18:00:04.467 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfa90 (RPC#1045) tag(415002d) sent to node 1 [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0415002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015609, gin 1, member 0 [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2311b20/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfa90 (RPC#1045) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdfa90, ret 0, client 0x23119b0 member 0x23111a0 [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x23119b0 [ CSSD]2013-05-23 18:00:04.468 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1045 [ CSSD]2013-05-23 18:00:04.474 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x23119b0) [ CSSD]2013-05-23 18:00:04.474 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x23119b0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:04.474 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x23111a0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:04.474 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1046 (0x7f431bfdfb10) [ CSSD]2013-05-23 18:00:04.474 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfb10 (RPC#1046) tag(416002d) sent to node 1 [ CSSD]2013-05-23 18:00:04.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0416002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:04.474 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:04.474 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfb10 (RPC#1046) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:04.474 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdfb10, ret 0, client 0x23119b0 member 0x23111a0 [ CSSD]2013-05-23 18:00:04.475 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1046 [ CSSD]2013-05-23 18:00:04.475 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x23111a0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:00:04.677 [1191196992] >TRACE: clssgmDeadProc: proc 0x23112c0 [ CSSD]2013-05-23 18:00:04.677 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x23112c0) con(0x2328e50) skgpid ospid 18306 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:04.751 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:05.437 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2328270) [ CSSD]2013-05-23 18:00:05.437 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2328270), grp DBHWPROD, member 1 [ CSSD]2013-05-23 18:00:05.437 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2328270), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 18:00:05.437 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 18:00:05.443 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x2328770) [ CSSD]2013-05-23 18:00:05.443 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x2328770), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:00:05.443 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x2328770), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:00:05.443 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 18:00:05.443 [1191196992] >TRACE: clssgmDeadProc: proc 0x2327d10 [ CSSD]2013-05-23 18:00:05.443 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2327d10) con(0x22e35d0) skgpid 18099 ospid 18099 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:05.491 [1191196992] >TRACE: clssgmAllocProc: (0x2329210) allocated [ CSSD]2013-05-23 18:00:05.492 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:00:05.493 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2329210 - 1,2,3 [ CSSD]2013-05-23 18:00:05.493 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f170) proc(0x2329210) pid(18353/18353) version 11:1:1:4 [ CSSD]2013-05-23 18:00:05.508 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x2329210), client(1/0x22ea2a0) [ CSSD]2013-05-23 18:00:05.508 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x22ea2a0) [ CSSD]2013-05-23 18:00:05.508 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:00:05.508 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 33 [ CSSD]2013-05-23 18:00:05.510 [1191196992] >TRACE: clssgmAllocProc: (0x2310ec0) allocated [ CSSD]2013-05-23 18:00:05.511 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:00:05.511 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x2310ec0 - 1,2,3 [ CSSD]2013-05-23 18:00:05.511 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2327f70) proc(0x2310ec0) pid(16806/16806) version 11:1:1:4 [ CSSD]2013-05-23 18:00:05.516 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2310ec0), client(1/0x23113f0) [ CSSD]2013-05-23 18:00:05.516 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x23113f0) [ CSSD]2013-05-23 18:00:05.516 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:00:05.516 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 34 [ CSSD]2013-05-23 18:00:06.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:06.164 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:06.210 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ea2a0) [ CSSD]2013-05-23 18:00:06.210 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ea2a0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:00:06.210 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ea2a0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:00:06.210 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 33 [ CSSD]2013-05-23 18:00:06.211 [1191196992] >TRACE: clssgmDeadProc: proc 0x2329210 [ CSSD]2013-05-23 18:00:06.211 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2329210) con(0x230f170) skgpid 18353 ospid 18353 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:06.271 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:06.271 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:06.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:07.585 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x2310ec0), client(2/0x22d7cd0) [ CSSD]2013-05-23 18:00:07.585 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22d7cd0) [ CSSD]2013-05-23 18:00:07.585 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 18:00:07.585 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 27 [ CSSD]2013-05-23 18:00:07.914 [1191196992] >TRACE: clssgmAllocProc: (0x22ea110) allocated [ CSSD]2013-05-23 18:00:07.915 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:00:07.916 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22ea110 - 1,2,3 [ CSSD]2013-05-23 18:00:07.916 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2328cc0) proc(0x22ea110) pid(18383) version 11:1:1:4 [ CSSD]2013-05-23 18:00:08.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:08.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:08.381 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ea110 [ CSSD]2013-05-23 18:00:08.381 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ea110) con(0x2328cc0) skgpid ospid 18383 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:08.767 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:09.214 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:00:09.214 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1047 (0x7f431bfdfb90) [ CSSD]2013-05-23 18:00:09.214 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfb90 (RPC#1047) tag(417002d) sent to node 1 [ CSSD]2013-05-23 18:00:09.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:09.215 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1047 (0x7f431bfdfb90) [ CSSD]2013-05-23 18:00:09.215 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfb90 (RPC#1047) state 6, flags 0x202 [ CSSD]2013-05-23 18:00:09.215 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfb90, ret 0, client (nil) [ CSSD]2013-05-23 18:00:09.215 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1047 [ CSSD]2013-05-23 18:00:10.222 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:10.222 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:10.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:11.274 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:11.274 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:12.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:12.228 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:12.779 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:14.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:14.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:14.550 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:14.551 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015610, gin 1, member 0 [ CSSD]2013-05-23 18:00:14.551 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x23288e0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:00:14.551 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:00:14.558 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:14.558 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:00:14.558 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x23112b0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:00:14.791 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:16.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:16.244 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:16.277 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:16.277 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:16.797 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:18.270 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:18.270 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:18.849 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:19.234 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:00:19.234 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1048 (0x7f431bfdfc10) [ CSSD]2013-05-23 18:00:19.235 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfc10 (RPC#1048) tag(418002d) sent to node 1 [ CSSD]2013-05-23 18:00:19.235 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:19.235 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1048 (0x7f431bfdfc10) [ CSSD]2013-05-23 18:00:19.235 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfc10 (RPC#1048) state 6, flags 0x202 [ CSSD]2013-05-23 18:00:19.235 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfc10, ret 0, client (nil) [ CSSD]2013-05-23 18:00:19.235 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1048 [ CSSD]2013-05-23 18:00:19.719 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c0e5840) allocated [ CSSD]2013-05-23 18:00:19.720 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:00:19.721 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c0e5840 - 1,2,3 [ CSSD]2013-05-23 18:00:19.721 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c0e5840) pid(18569) version 11:1:1:4 [ CSSD]2013-05-23 18:00:19.726 [1191196992] >TRACE: clssgmRegisterClient: proc(62/0x7f431c0e5840), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:19.726 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:19.726 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:00:19.726 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:19.726 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1049 (0x7f431bfdfc90) [ CSSD]2013-05-23 18:00:19.727 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfc90 (RPC#1049) tag(419002d) sent to node 1 [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0419002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015611, gin 1, member 0 [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfc90 (RPC#1049) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdfc90, ret 0, client 0x7f431c0e6e30 member 0x7f431c109eb0 [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:00:19.727 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1049 [ CSSD]2013-05-23 18:00:19.734 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:19.734 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:19.734 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c109eb0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:19.734 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1050 (0x7f431bfdfd10) [ CSSD]2013-05-23 18:00:19.734 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfd10 (RPC#1050) tag(41a002d) sent to node 1 [ CSSD]2013-05-23 18:00:19.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(041a002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:19.734 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:19.734 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfd10 (RPC#1050) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:19.734 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdfd10, ret 0, client 0x7f431c0e6e30 member 0x7f431c109eb0 [ CSSD]2013-05-23 18:00:19.734 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1050 [ CSSD]2013-05-23 18:00:19.735 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c109eb0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:00:19.937 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c0e5840 [ CSSD]2013-05-23 18:00:19.937 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c0e5840) con(0x7f431c11ab50) skgpid ospid 18569 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:20.323 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:20.323 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:20.808 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f4db0) [ CSSD]2013-05-23 18:00:20.808 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f4db0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:00:20.808 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f4db0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:00:20.808 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 32 [ CSSD]2013-05-23 18:00:20.808 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3000 [ CSSD]2013-05-23 18:00:20.808 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3000) con(0x22f2120) skgpid 13279 ospid 13279 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:20.824 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x205e370) [ CSSD]2013-05-23 18:00:20.824 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x205e370), grp DBHWPROD, member 1 [ CSSD]2013-05-23 18:00:20.824 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x205e370), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 18:00:20.824 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 26 [ CSSD]2013-05-23 18:00:20.834 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:00:20.834 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2312950), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:00:20.834 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2312950), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:00:20.834 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 18:00:20.834 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dfd50 [ CSSD]2013-05-23 18:00:20.834 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dfd50) con(0x231d6c0) skgpid 13281 ospid 13281 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:20.855 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:21.280 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:21.280 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:22.329 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:22.329 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:22.860 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:24.334 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:24.334 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:24.870 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:26.283 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:26.283 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:26.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:26.344 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:26.875 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:28.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:28.369 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:28.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:29.265 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:00:29.265 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1051 (0x7f431bfdfd90) [ CSSD]2013-05-23 18:00:29.265 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfd90 (RPC#1051) tag(41b002d) sent to node 1 [ CSSD]2013-05-23 18:00:29.266 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:29.266 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1051 (0x7f431bfdfd90) [ CSSD]2013-05-23 18:00:29.266 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfd90 (RPC#1051) state 6, flags 0x202 [ CSSD]2013-05-23 18:00:29.266 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdfd90, ret 0, client (nil) [ CSSD]2013-05-23 18:00:29.266 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1051 [ CSSD]2013-05-23 18:00:29.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:29.802 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015612, gin 1, member 0 [ CSSD]2013-05-23 18:00:29.802 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:00:29.802 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:00:29.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:29.809 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:00:29.809 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:00:30.285 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:30.285 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:00:30.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:30.373 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:30.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:32.398 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:32.398 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:32.901 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:34.404 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:34.404 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:34.969 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c121760) allocated [ CSSD]2013-05-23 18:00:34.970 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:00:34.970 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c121760 - 1,2,3 [ CSSD]2013-05-23 18:00:34.970 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c121760) pid(18710) version 11:1:1:4 [ CSSD]2013-05-23 18:00:34.971 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:34.975 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c121760), client(1/0x7f431c109f80) [ CSSD]2013-05-23 18:00:34.975 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 18:00:34.975 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c109f80 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 18:00:34.975 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:34.975 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1052 (0x7f431bfdfe10) [ CSSD]2013-05-23 18:00:34.976 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfe10 (RPC#1052) tag(41c002d) sent to node 1 [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(041c002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015613, gin 1, member 0 [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfe10 (RPC#1052) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdfe10, ret 0, client 0x7f431c109f80 member 0x7f431c0e6d60 [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c109f80 [ CSSD]2013-05-23 18:00:34.976 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1052 [ CSSD]2013-05-23 18:00:34.982 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c109f80) [ CSSD]2013-05-23 18:00:34.982 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c109f80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:34.982 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e6d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:34.982 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1053 (0x7f431bfdfe90) [ CSSD]2013-05-23 18:00:34.982 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdfe90 (RPC#1053) tag(41d002d) sent to node 1 [ CSSD]2013-05-23 18:00:34.983 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(041d002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:34.983 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:34.983 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdfe90 (RPC#1053) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:34.983 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfdfe90, ret 0, client 0x7f431c109f80 member 0x7f431c0e6d60 [ CSSD]2013-05-23 18:00:34.983 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1053 [ CSSD]2013-05-23 18:00:34.983 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e6d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:00:35.186 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c121760 [ CSSD]2013-05-23 18:00:35.186 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c121760) con(0x7f431c0e6720) skgpid ospid 18710 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:35.289 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:35.289 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:36.436 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:36.436 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:36.977 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:38.265 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 18:00:38.266 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:00:38.267 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 18:00:38.267 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c108da0) pid(18620/18620) version 11:1:1:4 [ CSSD]2013-05-23 18:00:38.283 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c108da0), client(1/0x7f431c0e5910) [ CSSD]2013-05-23 18:00:38.283 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 18:00:38.283 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:00:38.283 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 18:00:38.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:38.442 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:38.990 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:39.295 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:00:39.295 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1054 (0x7f431bfdff10) [ CSSD]2013-05-23 18:00:39.295 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdff10 (RPC#1054) tag(41e002d) sent to node 1 [ CSSD]2013-05-23 18:00:39.296 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:39.296 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1054 (0x7f431bfdff10) [ CSSD]2013-05-23 18:00:39.296 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdff10 (RPC#1054) state 6, flags 0x202 [ CSSD]2013-05-23 18:00:39.296 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfdff10, ret 0, client (nil) [ CSSD]2013-05-23 18:00:39.296 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1054 [ CSSD]2013-05-23 18:00:40.291 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:40.291 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:40.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:40.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:40.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:41.389 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:41.389 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015614, gin 1, member 0 [ CSSD]2013-05-23 18:00:41.389 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:00:41.389 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:00:41.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:41.396 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:00:41.396 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:00:42.479 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:42.479 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:43.000 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:44.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:44.485 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:45.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:45.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:45.060 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015615, gin 1, member 0 [ CSSD]2013-05-23 18:00:45.060 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:00:45.060 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:00:45.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:45.068 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:00:45.068 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:00:45.236 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:00:45.237 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:00:45.238 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:00:45.238 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c11fd30) pid(18838) version 11:1:1:4 [ CSSD]2013-05-23 18:00:45.243 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:45.243 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:45.243 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 18:00:45.243 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:45.243 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1055 (0x7f431bfdff90) [ CSSD]2013-05-23 18:00:45.243 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfdff90 (RPC#1055) tag(41f002d) sent to node 1 [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(041f002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015616, gin 1, member 0 [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfdff90 (RPC#1055) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfdff90, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:00:45.244 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1055 [ CSSD]2013-05-23 18:00:45.250 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:45.250 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:45.250 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:45.250 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1056 (0x7f431bfe0010) [ CSSD]2013-05-23 18:00:45.250 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0010 (RPC#1056) tag(420002d) sent to node 1 [ CSSD]2013-05-23 18:00:45.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0420002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:45.250 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:45.250 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0010 (RPC#1056) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:45.250 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0010, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:00:45.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1056 [ CSSD]2013-05-23 18:00:45.250 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:00:45.294 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:45.295 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:45.453 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:00:45.453 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e5200) skgpid ospid 18838 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:46.537 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:46.537 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:47.078 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:48.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:48.543 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:49.089 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:49.335 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:00:49.335 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1057 (0x7f431bfe0090) [ CSSD]2013-05-23 18:00:49.335 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0090 (RPC#1057) tag(421002d) sent to node 1 [ CSSD]2013-05-23 18:00:49.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:49.336 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1057 (0x7f431bfe0090) [ CSSD]2013-05-23 18:00:49.336 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0090 (RPC#1057) state 6, flags 0x202 [ CSSD]2013-05-23 18:00:49.336 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0090, ret 0, client (nil) [ CSSD]2013-05-23 18:00:49.336 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1057 [ CSSD]2013-05-23 18:00:50.226 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:00:50.227 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:00:50.227 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:00:50.227 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c11fd30) pid(18883) version 11:1:1:4 [ CSSD]2013-05-23 18:00:50.243 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:50.243 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:50.243 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 18:00:50.243 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:50.243 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1058 (0x7f431bfe0110) [ CSSD]2013-05-23 18:00:50.243 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0110 (RPC#1058) tag(422002d) sent to node 1 [ CSSD]2013-05-23 18:00:50.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0422002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:50.243 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015617, gin 1, member 0 [ CSSD]2013-05-23 18:00:50.243 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:00:50.243 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:00:50.243 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:00:50.244 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0110 (RPC#1058) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:50.244 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0110, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:00:50.244 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:00:50.244 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1058 [ CSSD]2013-05-23 18:00:50.249 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:00:50.249 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:50.249 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:00:50.249 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1059 (0x7f431bfe0190) [ CSSD]2013-05-23 18:00:50.249 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0190 (RPC#1059) tag(423002d) sent to node 1 [ CSSD]2013-05-23 18:00:50.250 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0423002d) incarnation(174631051) [ CSSD]2013-05-23 18:00:50.250 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:00:50.250 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0190 (RPC#1059) state 6, flags 0x100 [ CSSD]2013-05-23 18:00:50.250 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0190, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:00:50.250 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1059 [ CSSD]2013-05-23 18:00:50.250 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:00:50.298 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:50.298 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:50.453 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:00:50.453 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12f0c0) skgpid ospid 18883 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:00:50.573 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:50.573 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:51.094 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:52.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:52.579 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:53.100 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:54.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:54.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:55.153 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:55.301 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:00:55.301 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:00:56.636 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:56.636 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:57.159 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:58.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:58.680 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:00:59.171 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:00:59.365 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:00:59.365 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1060 (0x7f431bfe0210) [ CSSD]2013-05-23 18:00:59.365 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0210 (RPC#1060) tag(424002d) sent to node 1 [ CSSD]2013-05-23 18:00:59.366 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:00:59.366 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1060 (0x7f431bfe0210) [ CSSD]2013-05-23 18:00:59.366 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0210 (RPC#1060) state 6, flags 0x202 [ CSSD]2013-05-23 18:00:59.366 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0210, ret 0, client (nil) [ CSSD]2013-05-23 18:00:59.366 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1060 [ CSSD]2013-05-23 18:01:00.114 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 18:01:00.115 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:01:00.115 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 18:01:00.115 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e35d0) proc(0x22fc9c0) pid(19000) version 11:1:1:4 [ CSSD]2013-05-23 18:01:00.121 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22fc9c0), client(1/0x22f4e80) [ CSSD]2013-05-23 18:01:00.121 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4e80) [ CSSD]2013-05-23 18:01:00.121 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4e80 with con 0x2312940, requested num -1 [ CSSD]2013-05-23 18:01:00.121 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:00.121 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1061 (0x7f431bfe0290) [ CSSD]2013-05-23 18:01:00.121 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0290 (RPC#1061) tag(425002d) sent to node 1 [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0425002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015618, gin 1, member 0 [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0290 (RPC#1061) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0290, ret 0, client 0x22f4e80 member 0x22eec70 [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4e80 [ CSSD]2013-05-23 18:01:00.122 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1061 [ CSSD]2013-05-23 18:01:00.128 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4e80) [ CSSD]2013-05-23 18:01:00.128 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4e80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:00.128 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eec70) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:00.128 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1062 (0x7f431bfe0310) [ CSSD]2013-05-23 18:01:00.128 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0310 (RPC#1062) tag(426002d) sent to node 1 [ CSSD]2013-05-23 18:01:00.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0426002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:00.128 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:00.128 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0310 (RPC#1062) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:00.128 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0310, ret 0, client 0x22f4e80 member 0x22eec70 [ CSSD]2013-05-23 18:01:00.128 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1062 [ CSSD]2013-05-23 18:01:00.128 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eec70) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:00.304 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:00.304 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:00.312 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:00.312 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015619, gin 1, member 0 [ CSSD]2013-05-23 18:01:00.312 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:01:00.312 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:01:00.320 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:00.320 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:01:00.320 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:01:00.331 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 18:01:00.331 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x22e35d0) skgpid ospid 19000 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:00.735 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:00.735 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:01.177 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:02.741 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:02.741 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:03.183 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:04.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:04.746 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:05.204 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:05.307 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:05.308 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:05.483 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 18:01:05.485 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:01:05.485 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 18:01:05.485 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2f70) proc(0x22dacd0) pid(19058) version 11:1:1:4 [ CSSD]2013-05-23 18:01:05.491 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 18:01:05.491 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:01:05.491 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22fdc30, requested num -1 [ CSSD]2013-05-23 18:01:05.491 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:05.491 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1063 (0x7f431bfe0390) [ CSSD]2013-05-23 18:01:05.491 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0390 (RPC#1063) tag(427002d) sent to node 1 [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0427002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015620, gin 1, member 0 [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x2303160/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0390 (RPC#1063) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0390, ret 0, client 0x2312950 member 0x22e9650 [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 18:01:05.492 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1063 [ CSSD]2013-05-23 18:01:05.499 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:01:05.499 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:05.499 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22e9650) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:05.499 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1064 (0x7f431bfe0410) [ CSSD]2013-05-23 18:01:05.499 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0410 (RPC#1064) tag(428002d) sent to node 1 [ CSSD]2013-05-23 18:01:05.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0428002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:05.499 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:05.499 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0410 (RPC#1064) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:05.499 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0410, ret 0, client 0x2312950 member 0x22e9650 [ CSSD]2013-05-23 18:01:05.499 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1064 [ CSSD]2013-05-23 18:01:05.499 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22e9650) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:05.702 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 18:01:05.702 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x22e2f70) skgpid ospid 19058 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:06.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:06.758 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:07.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:08.764 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:08.764 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:09.262 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:09.395 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:01:09.395 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1065 (0x7f431bfe0490) [ CSSD]2013-05-23 18:01:09.395 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0490 (RPC#1065) tag(429002d) sent to node 1 [ CSSD]2013-05-23 18:01:09.396 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:09.396 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1065 (0x7f431bfe0490) [ CSSD]2013-05-23 18:01:09.396 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0490 (RPC#1065) state 6, flags 0x202 [ CSSD]2013-05-23 18:01:09.396 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0490, ret 0, client (nil) [ CSSD]2013-05-23 18:01:09.396 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1065 [ CSSD]2013-05-23 18:01:10.310 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:10.310 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:10.776 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:10.776 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:11.285 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:12.782 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:12.782 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:13.291 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:14.313 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:14.313 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:01:14.787 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:14.787 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:15.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:15.563 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:15.563 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015621, gin 1, member 0 [ CSSD]2013-05-23 18:01:15.563 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22f3b40/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:01:15.563 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:01:15.570 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:15.570 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:01:15.570 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:01:16.838 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:16.838 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:17.308 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:18.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:18.865 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:19.316 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:19.316 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:19.362 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:19.425 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:01:19.425 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1066 (0x7f431bfe0510) [ CSSD]2013-05-23 18:01:19.425 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0510 (RPC#1066) tag(42a002d) sent to node 1 [ CSSD]2013-05-23 18:01:19.426 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:19.426 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1066 (0x7f431bfe0510) [ CSSD]2013-05-23 18:01:19.426 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0510 (RPC#1066) state 6, flags 0x202 [ CSSD]2013-05-23 18:01:19.426 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0510, ret 0, client (nil) [ CSSD]2013-05-23 18:01:19.426 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1066 [ CSSD]2013-05-23 18:01:20.733 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:01:20.735 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:01:20.735 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:01:20.735 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c11fd30) pid(19209) version 11:1:1:4 [ CSSD]2013-05-23 18:01:20.741 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:01:20.741 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:01:20.741 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:01:20.741 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:20.741 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1067 (0x7f431bfe0590) [ CSSD]2013-05-23 18:01:20.741 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0590 (RPC#1067) tag(42b002d) sent to node 1 [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(042b002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015622, gin 1, member 0 [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0590 (RPC#1067) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0590, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:01:20.741 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1067 [ CSSD]2013-05-23 18:01:20.747 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:01:20.747 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:20.747 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:20.747 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1068 (0x7f431bfe0610) [ CSSD]2013-05-23 18:01:20.747 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0610 (RPC#1068) tag(42c002d) sent to node 1 [ CSSD]2013-05-23 18:01:20.747 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(042c002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:20.747 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:20.747 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0610 (RPC#1068) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:20.747 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0610, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:20.748 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1068 [ CSSD]2013-05-23 18:01:20.748 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:20.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:20.875 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:20.950 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:01:20.950 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12f0c0) skgpid ospid 19209 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:21.368 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:22.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:22.881 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:23.351 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:01:23.352 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:01:23.353 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:01:23.353 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c11fd30) pid(19280) version 11:1:1:4 [ CSSD]2013-05-23 18:01:23.360 [1191196992] >TRACE: clssgmRegisterClient: proc(61/0x7f431c11fd30), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:23.360 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:23.360 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:01:23.360 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 18:01:23.360 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1069 (0x7f431bfe0690) [ CSSD]2013-05-23 18:01:23.360 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0690 (RPC#1069) tag(42d002d) sent to node 1 [ CSSD]2013-05-23 18:01:23.360 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(042d002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:23.360 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015623, gin 1, member 0 [ CSSD]2013-05-23 18:01:23.360 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 18:01:23.361 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:23.361 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:23.361 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0690 (RPC#1069) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:23.361 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0690, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:01:23.361 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:01:23.361 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1069 [ CSSD]2013-05-23 18:01:23.367 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:23.367 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 18:01:23.367 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 18:01:23.367 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1070 (0x7f431bfe0710) [ CSSD]2013-05-23 18:01:23.367 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0710 (RPC#1070) tag(42e002d) sent to node 1 [ CSSD]2013-05-23 18:01:23.368 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(042e002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:23.368 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 18:01:23.368 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0710 (RPC#1070) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:23.368 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0710, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:01:23.368 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1070 [ CSSD]2013-05-23 18:01:23.368 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:23.374 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:23.482 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134600) allocated [ CSSD]2013-05-23 18:01:23.483 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:01:23.483 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134600 - 1,2,3 [ CSSD]2013-05-23 18:01:23.483 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135110) proc(0x7f431c134600) pid(19285) version 11:1:1:4 [ CSSD]2013-05-23 18:01:23.498 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:01:23.500 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:01:23.502 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134600 [ CSSD]2013-05-23 18:01:23.502 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134600) con(0x7f431c135110) skgpid ospid 19285 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:23.570 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:01:23.570 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c11ab50) skgpid ospid 19280 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:24.319 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:24.320 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:24.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:24.934 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:25.360 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22d7650) [ CSSD]2013-05-23 18:01:25.360 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22d7650), grp DBHWPROD, member 1 [ CSSD]2013-05-23 18:01:25.360 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22d7650), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 18:01:25.360 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 25 [ CSSD]2013-05-23 18:01:25.371 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:01:25.371 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:01:25.371 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f1d30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:01:25.371 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 18:01:25.371 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6cf0 [ CSSD]2013-05-23 18:01:25.371 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6cf0) con(0x22d7410) skgpid 13980 ospid 13980 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:25.385 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:26.568 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:01:26.568 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:01:26.569 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:01:26.569 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c11fd30) pid(19319/19319) version 11:1:1:4 [ CSSD]2013-05-23 18:01:26.575 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:26.575 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:26.575 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:01:26.575 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 32 [ CSSD]2013-05-23 18:01:26.587 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:26.587 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:01:26.587 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e6e30), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:01:26.587 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 31 [ CSSD]2013-05-23 18:01:26.587 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:01:26.587 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e6720) skgpid 19319 ospid 19319 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:26.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:26.939 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:27.391 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:28.965 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:28.965 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:29.323 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:29.323 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:29.402 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:29.445 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:01:29.445 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1071 (0x7f431bfe0790) [ CSSD]2013-05-23 18:01:29.445 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0790 (RPC#1071) tag(42f002d) sent to node 1 [ CSSD]2013-05-23 18:01:29.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:29.445 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1071 (0x7f431bfe0790) [ CSSD]2013-05-23 18:01:29.446 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0790 (RPC#1071) state 6, flags 0x202 [ CSSD]2013-05-23 18:01:29.446 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0790, ret 0, client (nil) [ CSSD]2013-05-23 18:01:29.446 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1071 [ CSSD]2013-05-23 18:01:30.814 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:30.815 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015624, gin 1, member 0 [ CSSD]2013-05-23 18:01:30.815 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:01:30.815 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:01:30.821 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:30.821 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:01:30.821 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:01:30.976 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:30.976 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:31.427 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:32.982 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:32.982 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:33.326 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:33.326 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:01:33.460 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:35.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:35.034 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:35.485 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:35.992 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:01:35.992 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:01:35.993 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:01:35.993 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c11fd30) pid(19411) version 11:1:1:4 [ CSSD]2013-05-23 18:01:35.998 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 18:01:35.998 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:01:35.998 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:01:35.998 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:35.998 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1072 (0x7f431bfe0810) [ CSSD]2013-05-23 18:01:35.999 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0810 (RPC#1072) tag(430002d) sent to node 1 [ CSSD]2013-05-23 18:01:35.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0430002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:35.999 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015625, gin 1, member 0 [ CSSD]2013-05-23 18:01:35.999 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:01:36.000 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:36.000 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:36.000 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0810 (RPC#1072) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:36.000 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0810, ret 0, client 0x7f431c12e6d0 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:36.000 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 18:01:36.000 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1072 [ CSSD]2013-05-23 18:01:36.006 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:01:36.006 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:36.006 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:36.006 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1073 (0x7f431bfe0890) [ CSSD]2013-05-23 18:01:36.006 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0890 (RPC#1073) tag(431002d) sent to node 1 [ CSSD]2013-05-23 18:01:36.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0431002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:36.006 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:36.006 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0890 (RPC#1073) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:36.006 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0890, ret 0, client 0x7f431c12e6d0 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:36.006 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1073 [ CSSD]2013-05-23 18:01:36.006 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:36.209 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:01:36.209 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12f0c0) skgpid ospid 19411 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:37.060 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:37.060 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:37.491 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:38.328 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:38.329 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:39.066 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:39.066 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:39.484 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:01:39.485 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1074 (0x7f431bfe0910) [ CSSD]2013-05-23 18:01:39.485 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0910 (RPC#1074) tag(432002d) sent to node 1 [ CSSD]2013-05-23 18:01:39.485 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:39.485 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1074 (0x7f431bfe0910) [ CSSD]2013-05-23 18:01:39.485 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0910 (RPC#1074) state 6, flags 0x202 [ CSSD]2013-05-23 18:01:39.485 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0910, ret 0, client (nil) [ CSSD]2013-05-23 18:01:39.486 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1074 [ CSSD]2013-05-23 18:01:39.503 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:41.077 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:41.077 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:41.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:41.645 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:41.645 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015626, gin 1, member 0 [ CSSD]2013-05-23 18:01:41.645 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:01:41.645 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:01:41.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:41.652 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:01:41.652 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:01:43.082 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:43.082 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:43.332 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:43.332 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:43.581 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:45.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:45.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:45.501 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:01:45.502 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:01:45.502 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:01:45.502 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c11fd30) pid(19517) version 11:1:1:4 [ CSSD]2013-05-23 18:01:45.508 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:01:45.508 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:01:45.508 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:01:45.508 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:45.508 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1075 (0x7f431bfe0990) [ CSSD]2013-05-23 18:01:45.508 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0990 (RPC#1075) tag(433002d) sent to node 1 [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0433002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015627, gin 1, member 0 [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0990 (RPC#1075) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0990, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:45.509 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:01:45.510 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1075 [ CSSD]2013-05-23 18:01:45.516 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:01:45.516 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:45.516 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:45.516 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1076 (0x7f431bfe0a10) [ CSSD]2013-05-23 18:01:45.516 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0a10 (RPC#1076) tag(434002d) sent to node 1 [ CSSD]2013-05-23 18:01:45.516 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0434002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:45.516 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:45.516 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0a10 (RPC#1076) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:45.516 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0a10, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:45.517 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1076 [ CSSD]2013-05-23 18:01:45.517 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:45.626 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:45.719 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:01:45.719 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e5200) skgpid ospid 19517 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:46.067 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:46.067 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015628, gin 1, member 0 [ CSSD]2013-05-23 18:01:46.067 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:01:46.067 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:01:46.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:46.074 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:01:46.074 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:01:47.160 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:47.160 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:47.632 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:48.335 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:48.335 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:49.166 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:49.166 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:49.516 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:01:49.516 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1077 (0x7f431bfe0a90) [ CSSD]2013-05-23 18:01:49.516 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0a90 (RPC#1077) tag(435002d) sent to node 1 [ CSSD]2013-05-23 18:01:49.517 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:49.517 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1077 (0x7f431bfe0a90) [ CSSD]2013-05-23 18:01:49.517 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0a90 (RPC#1077) state 6, flags 0x202 [ CSSD]2013-05-23 18:01:49.517 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0a90, ret 0, client (nil) [ CSSD]2013-05-23 18:01:49.517 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1077 [ CSSD]2013-05-23 18:01:49.664 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:51.247 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:01:51.247 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:01:51.248 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:01:51.248 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c11fd30) pid(19566) version 11:1:1:4 [ CSSD]2013-05-23 18:01:51.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:51.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:51.264 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:51.264 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:51.264 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:01:51.264 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:51.264 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1078 (0x7f431bfe0b10) [ CSSD]2013-05-23 18:01:51.264 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0b10 (RPC#1078) tag(436002d) sent to node 1 [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0436002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015629, gin 1, member 0 [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0b10 (RPC#1078) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0b10, ret 0, client 0x7f431c0e6e30 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:01:51.265 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1078 [ CSSD]2013-05-23 18:01:51.271 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:01:51.271 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:51.271 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:01:51.271 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1079 (0x7f431bfe0b90) [ CSSD]2013-05-23 18:01:51.271 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0b90 (RPC#1079) tag(437002d) sent to node 1 [ CSSD]2013-05-23 18:01:51.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0437002d) incarnation(174631051) [ CSSD]2013-05-23 18:01:51.271 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:01:51.271 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0b90 (RPC#1079) state 6, flags 0x100 [ CSSD]2013-05-23 18:01:51.271 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0b90, ret 0, client 0x7f431c0e6e30 member 0x7f431c12f520 [ CSSD]2013-05-23 18:01:51.271 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1079 [ CSSD]2013-05-23 18:01:51.271 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:01:51.474 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:01:51.474 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e5200) skgpid ospid 19566 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:01:51.669 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:52.338 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:52.338 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:01:53.264 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:53.264 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:53.761 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:55.346 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:55.346 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:55.787 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:57.341 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:01:57.341 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:01:57.351 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:57.351 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:57.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:01:59.358 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:59.358 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:01:59.545 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:01:59.546 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1080 (0x7f431bfe0c10) [ CSSD]2013-05-23 18:01:59.546 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0c10 (RPC#1080) tag(438002d) sent to node 1 [ CSSD]2013-05-23 18:01:59.546 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:01:59.546 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1080 (0x7f431bfe0c10) [ CSSD]2013-05-23 18:01:59.546 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0c10 (RPC#1080) state 6, flags 0x202 [ CSSD]2013-05-23 18:01:59.546 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0c10, ret 0, client (nil) [ CSSD]2013-05-23 18:01:59.547 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1080 [ CSSD]2013-05-23 18:01:59.804 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:01.318 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:01.318 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015630, gin 1, member 0 [ CSSD]2013-05-23 18:02:01.318 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:02:01.318 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:02:01.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:01.324 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:02:01.324 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:02:01.369 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:01.369 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:01.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:02.344 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:02.345 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:03.374 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:03.374 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:03.862 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:05.386 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:05.386 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:05.888 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:06.515 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:02:06.516 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:02:06.517 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:02:06.517 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c11fd30) pid(19736) version 11:1:1:4 [ CSSD]2013-05-23 18:02:06.523 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:06.523 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:06.523 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:02:06.523 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:06.523 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1081 (0x7f431bfe0c90) [ CSSD]2013-05-23 18:02:06.523 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0c90 (RPC#1081) tag(439002d) sent to node 1 [ CSSD]2013-05-23 18:02:06.523 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0439002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:06.523 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015631, gin 1, member 0 [ CSSD]2013-05-23 18:02:06.523 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:02:06.524 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:06.524 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:02:06.524 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0c90 (RPC#1081) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:06.524 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0c90, ret 0, client 0x7f431c12e6d0 member 0x7f431c12f520 [ CSSD]2013-05-23 18:02:06.524 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 18:02:06.524 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1081 [ CSSD]2013-05-23 18:02:06.530 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:06.530 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:06.530 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:06.530 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1082 (0x7f431bfe0d10) [ CSSD]2013-05-23 18:02:06.530 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0d10 (RPC#1082) tag(43a002d) sent to node 1 [ CSSD]2013-05-23 18:02:06.530 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(043a002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:06.530 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:06.530 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0d10 (RPC#1082) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:06.530 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0d10, ret 0, client 0x7f431c12e6d0 member 0x7f431c12f520 [ CSSD]2013-05-23 18:02:06.530 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1082 [ CSSD]2013-05-23 18:02:06.530 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:06.733 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:02:06.733 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c12f0c0) skgpid ospid 19736 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:07.348 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:07.348 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:07.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:07.391 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:07.894 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:09.416 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:09.416 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:09.576 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:02:09.577 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1083 (0x7f431bfe0d90) [ CSSD]2013-05-23 18:02:09.577 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0d90 (RPC#1083) tag(43b002d) sent to node 1 [ CSSD]2013-05-23 18:02:09.577 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:09.577 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1083 (0x7f431bfe0d90) [ CSSD]2013-05-23 18:02:09.577 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0d90 (RPC#1083) state 6, flags 0x202 [ CSSD]2013-05-23 18:02:09.577 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0d90, ret 0, client (nil) [ CSSD]2013-05-23 18:02:09.578 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1083 [ CSSD]2013-05-23 18:02:09.905 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:10.144 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:02:10.145 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:02:10.146 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:02:10.146 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c11fd30) pid(19806) version 11:1:1:4 [ CSSD]2013-05-23 18:02:10.152 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:02:10.152 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:02:10.152 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:02:10.152 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:10.152 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1084 (0x7f431bfe0e10) [ CSSD]2013-05-23 18:02:10.152 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0e10 (RPC#1084) tag(43c002d) sent to node 1 [ CSSD]2013-05-23 18:02:10.152 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(043c002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:10.152 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015632, gin 1, member 0 [ CSSD]2013-05-23 18:02:10.152 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:02:10.153 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:10.153 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:02:10.153 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0e10 (RPC#1084) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:10.153 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0e10, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 18:02:10.153 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:02:10.153 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1084 [ CSSD]2013-05-23 18:02:10.159 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:02:10.159 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:10.159 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:10.159 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1085 (0x7f431bfe0e90) [ CSSD]2013-05-23 18:02:10.159 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0e90 (RPC#1085) tag(43d002d) sent to node 1 [ CSSD]2013-05-23 18:02:10.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(043d002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:10.159 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:10.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0e90 (RPC#1085) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:10.159 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe0e90, ret 0, client 0x7f431c0cf720 member 0x7f431c12f520 [ CSSD]2013-05-23 18:02:10.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1085 [ CSSD]2013-05-23 18:02:10.159 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:10.362 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:02:10.362 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e6720) skgpid ospid 19806 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:11.351 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:11.351 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:02:11.449 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:11.449 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:11.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:13.453 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:13.453 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:13.962 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:15.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:15.466 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:15.969 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:16.354 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:16.354 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:16.580 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:16.580 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015633, gin 1, member 0 [ CSSD]2013-05-23 18:02:16.580 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:02:16.580 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:02:16.587 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:16.587 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:02:16.587 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:02:17.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:17.470 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:17.994 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:19.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:19.475 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:19.606 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:02:19.606 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1086 (0x7f431bfe0f10) [ CSSD]2013-05-23 18:02:19.606 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0f10 (RPC#1086) tag(43e002d) sent to node 1 [ CSSD]2013-05-23 18:02:19.607 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:19.607 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1086 (0x7f431bfe0f10) [ CSSD]2013-05-23 18:02:19.607 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0f10 (RPC#1086) state 6, flags 0x202 [ CSSD]2013-05-23 18:02:19.607 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe0f10, ret 0, client (nil) [ CSSD]2013-05-23 18:02:19.607 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1086 [ CSSD]2013-05-23 18:02:20.006 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:21.357 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:21.358 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:21.487 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:21.487 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:21.763 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:02:21.764 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:02:21.765 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:02:21.765 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c11fd30) pid(20094) version 11:1:1:4 [ CSSD]2013-05-23 18:02:21.770 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:21.770 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:21.770 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 18:02:21.770 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:21.770 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1087 (0x7f431bfe0f90) [ CSSD]2013-05-23 18:02:21.770 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe0f90 (RPC#1087) tag(43f002d) sent to node 1 [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(043f002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015634, gin 1, member 0 [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe0f90 (RPC#1087) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe0f90, ret 0, client 0x7f431c12e6d0 member 0x7f431c12f520 [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 18:02:21.771 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1087 [ CSSD]2013-05-23 18:02:21.777 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:21.777 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:21.777 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:21.777 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1088 (0x7f431bfe1010) [ CSSD]2013-05-23 18:02:21.777 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1010 (RPC#1088) tag(440002d) sent to node 1 [ CSSD]2013-05-23 18:02:21.777 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0440002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:21.778 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:21.778 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1010 (RPC#1088) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:21.778 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1010, ret 0, client 0x7f431c12e6d0 member 0x7f431c12f520 [ CSSD]2013-05-23 18:02:21.778 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1088 [ CSSD]2013-05-23 18:02:21.778 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:21.980 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:02:21.980 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e6720) skgpid ospid 20094 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:22.012 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:22.103 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:02:22.104 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:02:22.104 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:02:22.104 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c11fd30) pid(20141) version 11:1:1:4 [ CSSD]2013-05-23 18:02:22.235 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134600) allocated [ CSSD]2013-05-23 18:02:22.237 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:02:22.237 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134600 - 1,2,3 [ CSSD]2013-05-23 18:02:22.237 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12f0c0) proc(0x7f431c134600) pid(20151) version 11:1:1:4 [ CSSD]2013-05-23 18:02:22.253 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:02:22.255 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:02:22.257 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134600 [ CSSD]2013-05-23 18:02:22.257 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134600) con(0x7f431c12f0c0) skgpid ospid 20151 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:22.335 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(87/0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:22.335 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 87 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:22.335 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c12e6d0 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:02:22.335 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 18:02:22.335 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1089 (0x7f431bfe1090) [ CSSD]2013-05-23 18:02:22.335 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1090 (RPC#1089) tag(441002d) sent to node 1 [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0441002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015635, gin 1, member 0 [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c0958d0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1090 (RPC#1089) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1090, ret 0, client 0x7f431c12e6d0 member 0x7f431c13ea10 [ CSSD]2013-05-23 18:02:22.336 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 18:02:22.337 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1089 [ CSSD]2013-05-23 18:02:22.342 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 87 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:02:22.342 [1191196992] >TRACE: clssgmExitGrock: client 87 (0x7f431c12e6d0), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 18:02:22.342 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13ea10) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 18:02:22.342 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1090 (0x7f431bfe1110) [ CSSD]2013-05-23 18:02:22.342 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1110 (RPC#1090) tag(442002d) sent to node 1 [ CSSD]2013-05-23 18:02:22.343 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0442002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:22.343 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 18:02:22.343 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1110 (RPC#1090) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:22.343 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1110, ret 0, client 0x7f431c12e6d0 member 0x7f431c13ea10 [ CSSD]2013-05-23 18:02:22.343 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1090 [ CSSD]2013-05-23 18:02:22.343 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c13ea10) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:22.344 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:02:22.344 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c0e5200) skgpid ospid 20141 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:23.491 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:23.491 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:24.013 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:25.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:25.586 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:26.039 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:26.360 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:26.361 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:27.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:27.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:28.045 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:29.616 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:29.616 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:29.625 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:02:29.626 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1091 (0x7f431bfe1190) [ CSSD]2013-05-23 18:02:29.626 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1190 (RPC#1091) tag(443002d) sent to node 1 [ CSSD]2013-05-23 18:02:29.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:29.626 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1091 (0x7f431bfe1190) [ CSSD]2013-05-23 18:02:29.626 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1190 (RPC#1091) state 6, flags 0x202 [ CSSD]2013-05-23 18:02:29.626 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1190, ret 0, client (nil) [ CSSD]2013-05-23 18:02:29.627 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1091 [ CSSD]2013-05-23 18:02:30.076 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:30.363 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:30.363 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:02:31.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:31.668 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:31.833 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:31.833 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015636, gin 1, member 0 [ CSSD]2013-05-23 18:02:31.833 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:02:31.833 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:02:31.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:31.840 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:02:31.841 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:02:32.082 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:33.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:33.673 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:34.093 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:35.366 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:35.366 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:35.705 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:35.705 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:36.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:37.017 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c11fd30) allocated [ CSSD]2013-05-23 18:02:37.018 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:02:37.019 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c11fd30 - 1,2,3 [ CSSD]2013-05-23 18:02:37.019 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c11fd30) pid(20249) version 11:1:1:4 [ CSSD]2013-05-23 18:02:37.024 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x7f431c11fd30), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:02:37.024 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:02:37.024 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c12f0c0, requested num -1 [ CSSD]2013-05-23 18:02:37.024 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:37.024 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1092 (0x7f431bfe1210) [ CSSD]2013-05-23 18:02:37.024 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1210 (RPC#1092) tag(444002d) sent to node 1 [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0444002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015637, gin 1, member 0 [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1210 (RPC#1092) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1210, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:02:37.025 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1092 [ CSSD]2013-05-23 18:02:37.033 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:02:37.033 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:37.033 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:37.033 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1093 (0x7f431bfe1290) [ CSSD]2013-05-23 18:02:37.033 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1290 (RPC#1093) tag(445002d) sent to node 1 [ CSSD]2013-05-23 18:02:37.034 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0445002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:37.034 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:37.034 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1290 (RPC#1093) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:37.034 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1290, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:02:37.034 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1093 [ CSSD]2013-05-23 18:02:37.034 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:37.237 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c11fd30 [ CSSD]2013-05-23 18:02:37.237 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c11fd30) con(0x7f431c11ab50) skgpid ospid 20249 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:37.711 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:37.711 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:38.123 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:39.647 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:02:39.647 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1094 (0x7f431bfe1310) [ CSSD]2013-05-23 18:02:39.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1310 (RPC#1094) tag(446002d) sent to node 1 [ CSSD]2013-05-23 18:02:39.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:39.648 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1094 (0x7f431bfe1310) [ CSSD]2013-05-23 18:02:39.648 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1310 (RPC#1094) state 6, flags 0x202 [ CSSD]2013-05-23 18:02:39.648 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1310, ret 0, client (nil) [ CSSD]2013-05-23 18:02:39.648 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1094 [ CSSD]2013-05-23 18:02:39.718 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:39.718 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:40.196 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:40.369 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:40.370 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:41.750 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:41.750 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:41.906 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:41.906 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015638, gin 1, member 0 [ CSSD]2013-05-23 18:02:41.906 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:02:41.906 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:02:41.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:41.916 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:02:41.916 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:02:42.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:43.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:43.758 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:44.213 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:45.373 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:45.373 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:45.758 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 18:02:45.760 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:02:45.760 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 18:02:45.760 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2f70) proc(0x22dacd0) pid(20412) version 11:1:1:4 [ CSSD]2013-05-23 18:02:45.766 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22dacd0), client(1/0x22f4e80) [ CSSD]2013-05-23 18:02:45.766 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f4e80) [ CSSD]2013-05-23 18:02:45.766 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f4e80 with con 0x22fca90, requested num -1 [ CSSD]2013-05-23 18:02:45.766 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:45.766 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1095 (0x7f431bfe1390) [ CSSD]2013-05-23 18:02:45.767 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1390 (RPC#1095) tag(447002d) sent to node 1 [ CSSD]2013-05-23 18:02:45.767 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0447002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:45.767 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015639, gin 1, member 0 [ CSSD]2013-05-23 18:02:45.767 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:02:45.767 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:45.767 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:02:45.768 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1390 (RPC#1095) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:45.768 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1390, ret 0, client 0x22f4e80 member 0x22d7a00 [ CSSD]2013-05-23 18:02:45.768 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f4e80 [ CSSD]2013-05-23 18:02:45.768 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1095 [ CSSD]2013-05-23 18:02:45.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:45.768 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:45.774 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f4e80) [ CSSD]2013-05-23 18:02:45.774 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f4e80), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:45.774 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7a00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:45.774 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1096 (0x7f431bfe1410) [ CSSD]2013-05-23 18:02:45.774 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1410 (RPC#1096) tag(448002d) sent to node 1 [ CSSD]2013-05-23 18:02:45.775 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0448002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:45.775 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:45.775 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1410 (RPC#1096) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:45.775 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1410, ret 0, client 0x22f4e80 member 0x22d7a00 [ CSSD]2013-05-23 18:02:45.775 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1096 [ CSSD]2013-05-23 18:02:45.775 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7a00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:45.978 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 18:02:45.978 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x22e2f70) skgpid ospid 20412 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:46.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:47.084 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:47.084 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015640, gin 1, member 0 [ CSSD]2013-05-23 18:02:47.085 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:02:47.085 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:02:47.092 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:47.092 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:02:47.092 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22f4db0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:02:47.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:47.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:48.225 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:49.666 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:02:49.667 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1097 (0x7f431bfe1490) [ CSSD]2013-05-23 18:02:49.667 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1490 (RPC#1097) tag(449002d) sent to node 1 [ CSSD]2013-05-23 18:02:49.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:49.668 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1097 (0x7f431bfe1490) [ CSSD]2013-05-23 18:02:49.668 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1490 (RPC#1097) state 6, flags 0x202 [ CSSD]2013-05-23 18:02:49.668 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1490, ret 0, client (nil) [ CSSD]2013-05-23 18:02:49.668 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1097 [ CSSD]2013-05-23 18:02:49.780 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:49.780 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:50.277 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:50.376 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:50.376 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:50.842 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x2321f80) [ CSSD]2013-05-23 18:02:50.842 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x2321f80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:02:50.842 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x2321f80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:02:50.842 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 30 [ CSSD]2013-05-23 18:02:50.842 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e53b0 [ CSSD]2013-05-23 18:02:50.842 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e53b0) con(0x22e3c90) skgpid 14962 ospid 14962 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:50.844 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22ee170) [ CSSD]2013-05-23 18:02:50.844 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22ee170), grp DBHWPROD, member 1 [ CSSD]2013-05-23 18:02:50.844 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22ee170), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 18:02:50.844 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 24 [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22e9790) [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22e9790), grp DBHWPROD, member 1 [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22e9790), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 23 [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22ede80) [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22ede80), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22ede80), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:02:50.854 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 18:02:50.855 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ed960 [ CSSD]2013-05-23 18:02:50.855 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ed960) con(0x22f4f20) skgpid 14956 ospid 14956 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:50.858 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22e9290) [ CSSD]2013-05-23 18:02:50.858 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22e9290), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:02:50.858 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22e9290), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:02:50.858 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 18:02:50.858 [1191196992] >TRACE: clssgmDeadProc: proc 0x22e8ca0 [ CSSD]2013-05-23 18:02:50.858 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22e8ca0) con(0x2322680) skgpid 14960 ospid 14960 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:51.791 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:51.791 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:52.269 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 18:02:52.270 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:02:52.271 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 18:02:52.271 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22e2f80) proc(0x22dacd0) pid(20494) version 11:1:1:4 [ CSSD]2013-05-23 18:02:52.277 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x22dff40) [ CSSD]2013-05-23 18:02:52.277 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22dff40) [ CSSD]2013-05-23 18:02:52.277 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22dff40 with con 0x231d790, requested num -1 [ CSSD]2013-05-23 18:02:52.277 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:52.277 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1098 (0x7f431bfe1510) [ CSSD]2013-05-23 18:02:52.277 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1510 (RPC#1098) tag(44a002d) sent to node 1 [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(044a002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015641, gin 1, member 0 [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1510 (RPC#1098) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1510, ret 0, client 0x22dff40 member 0x22eed40 [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22dff40 [ CSSD]2013-05-23 18:02:52.278 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1098 [ CSSD]2013-05-23 18:02:52.283 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:52.284 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22dff40) [ CSSD]2013-05-23 18:02:52.284 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22dff40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:52.284 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eed40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:02:52.284 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1099 (0x7f431bfe1590) [ CSSD]2013-05-23 18:02:52.285 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1590 (RPC#1099) tag(44b002d) sent to node 1 [ CSSD]2013-05-23 18:02:52.285 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(044b002d) incarnation(174631051) [ CSSD]2013-05-23 18:02:52.285 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:02:52.285 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1590 (RPC#1099) state 6, flags 0x100 [ CSSD]2013-05-23 18:02:52.285 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1590, ret 0, client 0x22dff40 member 0x22eed40 [ CSSD]2013-05-23 18:02:52.285 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1099 [ CSSD]2013-05-23 18:02:52.285 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eed40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:02:52.488 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 18:02:52.488 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x22e2f80) skgpid ospid 20494 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:02:53.817 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:53.817 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:54.295 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:55.379 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:02:55.380 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:02:55.869 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:55.869 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:56.320 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:57.875 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:57.875 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:02:58.326 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:02:59.697 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:02:59.697 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1100 (0x7f431bfe1610) [ CSSD]2013-05-23 18:02:59.697 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1610 (RPC#1100) tag(44c002d) sent to node 1 [ CSSD]2013-05-23 18:02:59.698 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:59.698 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1100 (0x7f431bfe1610) [ CSSD]2013-05-23 18:02:59.698 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1610 (RPC#1100) state 6, flags 0x202 [ CSSD]2013-05-23 18:02:59.698 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1610, ret 0, client (nil) [ CSSD]2013-05-23 18:02:59.698 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1100 [ CSSD]2013-05-23 18:02:59.882 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:02:59.882 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:00.378 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:00.383 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:00.383 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:01.891 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:01.891 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:02.336 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:02.336 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015642, gin 1, member 0 [ CSSD]2013-05-23 18:03:02.336 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:03:02.336 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:03:02.344 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:02.344 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:03:02.344 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2312880) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:03:02.384 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:03.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:03.898 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:04.394 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:05.386 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:05.386 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:05.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:05.968 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:06.419 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:07.519 [1191196992] >TRACE: clssgmAllocProc: (0x22d7a00) allocated [ CSSD]2013-05-23 18:03:07.521 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:03:07.521 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7a00 - 1,2,3 [ CSSD]2013-05-23 18:03:07.521 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x231d6c0) proc(0x22d7a00) pid(20645) version 11:1:1:4 [ CSSD]2013-05-23 18:03:07.527 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7a00), client(1/0x2312950) [ CSSD]2013-05-23 18:03:07.527 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:03:07.527 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22dada0, requested num -1 [ CSSD]2013-05-23 18:03:07.528 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:07.528 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1101 (0x7f431bfe1690) [ CSSD]2013-05-23 18:03:07.528 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1690 (RPC#1101) tag(44d002d) sent to node 1 [ CSSD]2013-05-23 18:03:07.528 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(044d002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:07.528 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015643, gin 1, member 0 [ CSSD]2013-05-23 18:03:07.528 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22d7350/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:03:07.528 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:03:07.529 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:03:07.529 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1690 (RPC#1101) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:07.529 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1690, ret 0, client 0x2312950 member 0x22eed40 [ CSSD]2013-05-23 18:03:07.529 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 18:03:07.529 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1101 [ CSSD]2013-05-23 18:03:07.535 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:03:07.536 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:07.536 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22eed40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:07.536 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1102 (0x7f431bfe1710) [ CSSD]2013-05-23 18:03:07.536 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1710 (RPC#1102) tag(44e002d) sent to node 1 [ CSSD]2013-05-23 18:03:07.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(044e002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:07.536 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:07.536 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1710 (RPC#1102) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:07.536 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1710, ret 0, client 0x2312950 member 0x22eed40 [ CSSD]2013-05-23 18:03:07.536 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1102 [ CSSD]2013-05-23 18:03:07.536 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22eed40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:03:07.739 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7a00 [ CSSD]2013-05-23 18:03:07.739 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7a00) con(0x231d6c0) skgpid ospid 20645 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:03:07.973 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:07.973 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:08.473 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:09.389 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:09.389 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:03:09.727 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:03:09.727 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1103 (0x7f431bfe1790) [ CSSD]2013-05-23 18:03:09.727 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1790 (RPC#1103) tag(44f002d) sent to node 1 [ CSSD]2013-05-23 18:03:09.728 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:09.728 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1103 (0x7f431bfe1790) [ CSSD]2013-05-23 18:03:09.728 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1790 (RPC#1103) state 6, flags 0x202 [ CSSD]2013-05-23 18:03:09.728 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1790, ret 0, client (nil) [ CSSD]2013-05-23 18:03:09.728 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1103 [ CSSD]2013-05-23 18:03:09.985 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:09.985 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:10.479 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:11.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:11.991 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:12.484 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:13.997 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:13.997 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:14.392 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:14.392 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:14.576 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:16.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:16.020 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:16.601 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:17.589 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:17.589 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015644, gin 1, member 0 [ CSSD]2013-05-23 18:03:17.589 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:03:17.589 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:03:17.595 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:17.596 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:03:17.596 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22e2eb0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:03:18.026 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:18.026 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:18.612 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:19.254 [1191196992] >TRACE: clssgmAllocProc: (0x22d7a00) allocated [ CSSD]2013-05-23 18:03:19.255 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:03:19.256 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7a00 - 1,2,3 [ CSSD]2013-05-23 18:03:19.256 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dacd0) proc(0x22d7a00) pid(19308/19308) version 11:1:1:4 [ CSSD]2013-05-23 18:03:19.262 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7a00), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 18:03:19.262 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:03:19.262 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:03:19.262 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 18:03:19.395 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:19.395 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:19.757 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:03:19.757 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1104 (0x7f431bfe1810) [ CSSD]2013-05-23 18:03:19.757 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1810 (RPC#1104) tag(450002d) sent to node 1 [ CSSD]2013-05-23 18:03:19.758 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:19.758 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1104 (0x7f431bfe1810) [ CSSD]2013-05-23 18:03:19.758 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1810 (RPC#1104) state 6, flags 0x202 [ CSSD]2013-05-23 18:03:19.758 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1810, ret 0, client (nil) [ CSSD]2013-05-23 18:03:19.758 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1104 [ CSSD]2013-05-23 18:03:20.057 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:20.057 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:20.618 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:22.064 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:22.064 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:22.623 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:22.770 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:03:22.771 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:03:22.772 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:03:22.772 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135110) proc(0x7f431c108370) pid(20816) version 11:1:1:4 [ CSSD]2013-05-23 18:03:22.777 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:03:22.777 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:03:22.777 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:03:22.777 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:22.777 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1105 (0x7f431bfe1890) [ CSSD]2013-05-23 18:03:22.777 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1890 (RPC#1105) tag(451002d) sent to node 1 [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0451002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015645, gin 1, member 0 [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1890 (RPC#1105) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1890, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:03:22.778 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1105 [ CSSD]2013-05-23 18:03:22.784 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:03:22.784 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:22.784 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:22.784 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1106 (0x7f431bfe1910) [ CSSD]2013-05-23 18:03:22.784 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1910 (RPC#1106) tag(452002d) sent to node 1 [ CSSD]2013-05-23 18:03:22.785 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0452002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:22.785 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:22.785 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1910 (RPC#1106) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:22.785 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1910, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 18:03:22.785 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1106 [ CSSD]2013-05-23 18:03:22.785 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:03:22.988 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:03:22.988 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c135110) skgpid ospid 20816 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:03:24.090 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:24.090 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:24.397 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:24.398 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:24.675 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:26.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:26.120 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:26.701 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:28.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:28.126 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:28.714 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:29.401 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:29.401 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:29.778 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:03:29.778 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1107 (0x7f431bfe1990) [ CSSD]2013-05-23 18:03:29.778 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1990 (RPC#1107) tag(453002d) sent to node 1 [ CSSD]2013-05-23 18:03:29.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:29.779 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1107 (0x7f431bfe1990) [ CSSD]2013-05-23 18:03:29.779 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1990 (RPC#1107) state 6, flags 0x202 [ CSSD]2013-05-23 18:03:29.779 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1990, ret 0, client (nil) [ CSSD]2013-05-23 18:03:29.779 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1107 [ CSSD]2013-05-23 18:03:30.158 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:30.158 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:30.719 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:32.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:32.164 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:32.725 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:32.840 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:32.840 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015646, gin 1, member 0 [ CSSD]2013-05-23 18:03:32.840 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:03:32.840 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:03:32.846 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:32.846 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:03:32.846 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:03:33.404 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:33.404 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:03:34.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:34.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:34.777 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:36.181 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:36.181 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:36.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:38.030 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:03:38.031 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:03:38.031 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:03:38.031 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c108370) pid(20968) version 11:1:1:4 [ CSSD]2013-05-23 18:03:38.037 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:03:38.037 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:03:38.037 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c135110, requested num -1 [ CSSD]2013-05-23 18:03:38.037 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:38.037 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1108 (0x7f431bfe1a10) [ CSSD]2013-05-23 18:03:38.038 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1a10 (RPC#1108) tag(454002d) sent to node 1 [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0454002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015647, gin 1, member 0 [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1a10 (RPC#1108) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1a10, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:03:38.038 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:03:38.039 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1108 [ CSSD]2013-05-23 18:03:38.045 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:03:38.045 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:38.045 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:38.045 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1109 (0x7f431bfe1a90) [ CSSD]2013-05-23 18:03:38.045 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1a90 (RPC#1109) tag(455002d) sent to node 1 [ CSSD]2013-05-23 18:03:38.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0455002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:38.045 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:38.045 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1a90 (RPC#1109) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:38.045 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1a90, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:03:38.045 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1109 [ CSSD]2013-05-23 18:03:38.045 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:03:38.187 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:38.187 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:38.248 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:03:38.248 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11ab50) skgpid ospid 20968 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:03:38.407 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:38.407 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:38.813 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:39.798 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:03:39.798 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1110 (0x7f431bfe1b10) [ CSSD]2013-05-23 18:03:39.798 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1b10 (RPC#1110) tag(456002d) sent to node 1 [ CSSD]2013-05-23 18:03:39.799 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:39.799 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1110 (0x7f431bfe1b10) [ CSSD]2013-05-23 18:03:39.799 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1b10 (RPC#1110) state 6, flags 0x202 [ CSSD]2013-05-23 18:03:39.799 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1b10, ret 0, client (nil) [ CSSD]2013-05-23 18:03:39.799 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1110 [ CSSD]2013-05-23 18:03:40.258 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:40.258 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:40.819 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:42.156 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:42.156 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015648, gin 1, member 0 [ CSSD]2013-05-23 18:03:42.156 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:03:42.156 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:03:42.163 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:42.163 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:03:42.163 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:03:42.284 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:42.284 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:42.824 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:43.410 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:43.410 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:44.291 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:44.291 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:44.656 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:44.656 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015649, gin 1, member 0 [ CSSD]2013-05-23 18:03:44.656 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 18:03:44.656 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:03:44.662 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:44.662 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 18:03:44.662 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:03:44.787 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:03:44.788 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:03:44.789 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:03:44.790 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:03:44.791 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:03:44.791 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:03:44.877 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:46.020 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 18:03:46.021 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:03:46.021 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 18:03:46.021 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23112a0) proc(0x22d7350) pid(21064) version 11:1:1:4 [ CSSD]2013-05-23 18:03:46.027 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7350), client(1/0x2311350) [ CSSD]2013-05-23 18:03:46.027 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2311350) [ CSSD]2013-05-23 18:03:46.027 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2311350 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 18:03:46.027 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:46.027 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1111 (0x7f431bfe1b90) [ CSSD]2013-05-23 18:03:46.028 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1b90 (RPC#1111) tag(457002d) sent to node 1 [ CSSD]2013-05-23 18:03:46.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0457002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:46.028 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015650, gin 1, member 0 [ CSSD]2013-05-23 18:03:46.028 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:03:46.028 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:03:46.029 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:03:46.029 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1b90 (RPC#1111) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:46.029 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1b90, ret 0, client 0x2311350 member 0x22d7550 [ CSSD]2013-05-23 18:03:46.029 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2311350 [ CSSD]2013-05-23 18:03:46.029 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1111 [ CSSD]2013-05-23 18:03:46.034 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2311350) [ CSSD]2013-05-23 18:03:46.034 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2311350), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:46.034 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7550) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:46.034 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1112 (0x7f431bfe1c10) [ CSSD]2013-05-23 18:03:46.034 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1c10 (RPC#1112) tag(458002d) sent to node 1 [ CSSD]2013-05-23 18:03:46.035 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0458002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:46.035 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:46.035 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1c10 (RPC#1112) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:46.035 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1c10, ret 0, client 0x2311350 member 0x22d7550 [ CSSD]2013-05-23 18:03:46.035 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1112 [ CSSD]2013-05-23 18:03:46.035 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7550) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:03:46.238 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 18:03:46.238 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x23112a0) skgpid ospid 21064 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:03:46.300 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:46.300 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:46.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:48.097 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:48.097 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015651, gin 1, member 0 [ CSSD]2013-05-23 18:03:48.097 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:03:48.097 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:03:48.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:48.104 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:03:48.104 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:03:48.307 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:48.307 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:48.414 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:48.414 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:48.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:49.818 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:03:49.818 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1113 (0x7f431bfe1c90) [ CSSD]2013-05-23 18:03:49.818 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1c90 (RPC#1113) tag(459002d) sent to node 1 [ CSSD]2013-05-23 18:03:49.819 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:49.819 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1113 (0x7f431bfe1c90) [ CSSD]2013-05-23 18:03:49.819 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1c90 (RPC#1113) state 6, flags 0x202 [ CSSD]2013-05-23 18:03:49.819 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1c90, ret 0, client (nil) [ CSSD]2013-05-23 18:03:49.819 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1113 [ CSSD]2013-05-23 18:03:50.359 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:50.359 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:50.921 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:52.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:52.365 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:52.416 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:52.417 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:03:52.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:53.279 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 18:03:53.280 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:03:53.281 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 18:03:53.281 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23112a0) proc(0x22d7350) pid(21120) version 11:1:1:4 [ CSSD]2013-05-23 18:03:53.286 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7350), client(1/0x2311350) [ CSSD]2013-05-23 18:03:53.286 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2311350) [ CSSD]2013-05-23 18:03:53.286 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2311350 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 18:03:53.286 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:53.286 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1114 (0x7f431bfe1d10) [ CSSD]2013-05-23 18:03:53.286 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1d10 (RPC#1114) tag(45a002d) sent to node 1 [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(045a002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015652, gin 1, member 0 [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1d10 (RPC#1114) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1d10, ret 0, client 0x2311350 member 0x22d7550 [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2311350 [ CSSD]2013-05-23 18:03:53.287 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1114 [ CSSD]2013-05-23 18:03:53.293 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2311350) [ CSSD]2013-05-23 18:03:53.293 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2311350), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:53.293 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7550) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:03:53.293 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1115 (0x7f431bfe1d90) [ CSSD]2013-05-23 18:03:53.293 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1d90 (RPC#1115) tag(45b002d) sent to node 1 [ CSSD]2013-05-23 18:03:53.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(045b002d) incarnation(174631051) [ CSSD]2013-05-23 18:03:53.293 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:03:53.293 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1d90 (RPC#1115) state 6, flags 0x100 [ CSSD]2013-05-23 18:03:53.293 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1d90, ret 0, client 0x2311350 member 0x22d7550 [ CSSD]2013-05-23 18:03:53.294 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1115 [ CSSD]2013-05-23 18:03:53.294 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7550) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:03:53.496 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 18:03:53.496 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x23112a0) skgpid ospid 21120 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:03:54.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:54.372 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:54.927 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:56.382 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:56.382 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:56.953 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:57.420 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:03:57.420 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:03:58.388 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:58.388 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:03:58.985 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:03:59.858 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:03:59.858 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1116 (0x7f431bfe1e10) [ CSSD]2013-05-23 18:03:59.858 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1e10 (RPC#1116) tag(45c002d) sent to node 1 [ CSSD]2013-05-23 18:03:59.859 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:03:59.859 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1116 (0x7f431bfe1e10) [ CSSD]2013-05-23 18:03:59.859 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1e10 (RPC#1116) state 6, flags 0x202 [ CSSD]2013-05-23 18:03:59.859 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1e10, ret 0, client (nil) [ CSSD]2013-05-23 18:03:59.859 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1116 [ CSSD]2013-05-23 18:04:00.399 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:00.399 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:00.992 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:02.423 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:02.423 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:02.445 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:02.445 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:02.998 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:03.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:03.356 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015653, gin 1, member 0 [ CSSD]2013-05-23 18:04:03.356 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:04:03.356 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:04:03.363 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:03.363 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:04:03.363 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:04:04.452 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:04.452 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:05.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:06.462 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:06.462 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:07.053 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:07.426 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:07.426 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:08.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:08.469 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:08.528 [1191196992] >TRACE: clssgmAllocProc: (0x231ed30) allocated [ CSSD]2013-05-23 18:04:08.529 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:04:08.530 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x231ed30 - 1,2,3 [ CSSD]2013-05-23 18:04:08.530 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23111e0) proc(0x231ed30) pid(21270) version 11:1:1:4 [ CSSD]2013-05-23 18:04:08.536 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x231ed30), client(1/0x22f1d30) [ CSSD]2013-05-23 18:04:08.536 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:04:08.536 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 18:04:08.536 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:08.536 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1117 (0x7f431bfe1e90) [ CSSD]2013-05-23 18:04:08.536 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1e90 (RPC#1117) tag(45d002d) sent to node 1 [ CSSD]2013-05-23 18:04:08.536 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(045d002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:08.536 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015654, gin 1, member 0 [ CSSD]2013-05-23 18:04:08.536 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:04:08.537 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:04:08.537 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:04:08.537 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1e90 (RPC#1117) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:08.537 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe1e90, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 18:04:08.537 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:04:08.537 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1117 [ CSSD]2013-05-23 18:04:08.543 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:04:08.543 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:08.543 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231eb40) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:08.543 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1118 (0x7f431bfe1f10) [ CSSD]2013-05-23 18:04:08.543 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1f10 (RPC#1118) tag(45e002d) sent to node 1 [ CSSD]2013-05-23 18:04:08.543 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(045e002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:08.543 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:08.543 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1f10 (RPC#1118) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:08.543 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe1f10, ret 0, client 0x22f1d30 member 0x231eb40 [ CSSD]2013-05-23 18:04:08.544 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1118 [ CSSD]2013-05-23 18:04:08.544 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231eb40) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:04:08.746 [1191196992] >TRACE: clssgmDeadProc: proc 0x231ed30 [ CSSD]2013-05-23 18:04:08.746 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x231ed30) con(0x23111e0) skgpid ospid 21270 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:04:09.086 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:09.888 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:04:09.888 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1119 (0x7f431bfe1f90) [ CSSD]2013-05-23 18:04:09.888 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe1f90 (RPC#1119) tag(45f002d) sent to node 1 [ CSSD]2013-05-23 18:04:09.889 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:09.889 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1119 (0x7f431bfe1f90) [ CSSD]2013-05-23 18:04:09.889 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe1f90 (RPC#1119) state 6, flags 0x202 [ CSSD]2013-05-23 18:04:09.889 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe1f90, ret 0, client (nil) [ CSSD]2013-05-23 18:04:09.889 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1119 [ CSSD]2013-05-23 18:04:10.480 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:10.480 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:11.091 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:11.429 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:11.429 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:04:12.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:12.525 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:13.098 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:14.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:14.532 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:15.109 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:16.432 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:16.432 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:16.603 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:16.603 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:17.115 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:18.608 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:18.608 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015655, gin 1, member 0 [ CSSD]2013-05-23 18:04:18.608 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:04:18.608 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:04:18.614 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:18.614 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:04:18.614 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:04:18.628 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:18.628 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:19.127 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:19.916 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:04:19.916 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1120 (0x7f431bfe2010) [ CSSD]2013-05-23 18:04:19.916 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2010 (RPC#1120) tag(460002d) sent to node 1 [ CSSD]2013-05-23 18:04:19.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:19.917 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1120 (0x7f431bfe2010) [ CSSD]2013-05-23 18:04:19.917 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2010 (RPC#1120) state 6, flags 0x202 [ CSSD]2013-05-23 18:04:19.917 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2010, ret 0, client (nil) [ CSSD]2013-05-23 18:04:19.917 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1120 [ CSSD]2013-05-23 18:04:20.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:20.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:21.153 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:21.435 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:21.435 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:22.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:22.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:23.158 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:23.788 [1191196992] >TRACE: clssgmAllocProc: (0x22d7350) allocated [ CSSD]2013-05-23 18:04:23.789 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:04:23.789 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7350 - 1,2,3 [ CSSD]2013-05-23 18:04:23.789 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x23112a0) proc(0x22d7350) pid(21428) version 11:1:1:4 [ CSSD]2013-05-23 18:04:23.795 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d7350), client(1/0x2311350) [ CSSD]2013-05-23 18:04:23.795 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2311350) [ CSSD]2013-05-23 18:04:23.795 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2311350 with con 0x22dffa0, requested num -1 [ CSSD]2013-05-23 18:04:23.795 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:23.795 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1121 (0x7f431bfe2090) [ CSSD]2013-05-23 18:04:23.795 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2090 (RPC#1121) tag(461002d) sent to node 1 [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0461002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015656, gin 1, member 0 [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f37a0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2090 (RPC#1121) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2090, ret 0, client 0x2311350 member 0x22d7550 [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2311350 [ CSSD]2013-05-23 18:04:23.796 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1121 [ CSSD]2013-05-23 18:04:23.802 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2311350) [ CSSD]2013-05-23 18:04:23.802 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2311350), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:23.802 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d7550) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:23.802 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1122 (0x7f431bfe2110) [ CSSD]2013-05-23 18:04:23.802 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2110 (RPC#1122) tag(462002d) sent to node 1 [ CSSD]2013-05-23 18:04:23.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0462002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:23.802 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:23.802 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2110 (RPC#1122) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:23.802 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2110, ret 0, client 0x2311350 member 0x22d7550 [ CSSD]2013-05-23 18:04:23.803 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1122 [ CSSD]2013-05-23 18:04:23.803 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d7550) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:04:24.005 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7350 [ CSSD]2013-05-23 18:04:24.005 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7350) con(0x23112a0) skgpid ospid 21428 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:04:24.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:24.694 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:25.190 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:26.438 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:26.438 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:26.704 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:26.704 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:27.195 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:28.710 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:28.710 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:29.207 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:29.950 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:04:29.951 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1123 (0x7f431bfe2190) [ CSSD]2013-05-23 18:04:29.951 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2190 (RPC#1123) tag(463002d) sent to node 1 [ CSSD]2013-05-23 18:04:29.952 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:29.952 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1123 (0x7f431bfe2190) [ CSSD]2013-05-23 18:04:29.952 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2190 (RPC#1123) state 6, flags 0x202 [ CSSD]2013-05-23 18:04:29.952 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2190, ret 0, client (nil) [ CSSD]2013-05-23 18:04:29.952 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1123 [ CSSD]2013-05-23 18:04:30.441 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:30.441 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:04:30.781 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:30.782 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:31.212 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:32.118 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:04:32.119 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:04:32.119 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:04:32.120 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:04:32.195 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:32.195 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015657, gin 1, member 0 [ CSSD]2013-05-23 18:04:32.195 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x22dfd50/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 18:04:32.195 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 18:04:32.202 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:32.202 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 18:04:32.202 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:04:32.788 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:32.788 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:33.219 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:33.865 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:33.865 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015658, gin 1, member 0 [ CSSD]2013-05-23 18:04:33.865 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:04:33.865 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:04:33.872 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:33.873 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:04:33.873 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:04:34.794 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:34.794 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:35.229 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:35.444 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:35.444 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:36.803 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:36.803 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:37.235 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:38.809 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:38.809 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:39.037 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:04:39.038 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:04:39.039 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:04:39.039 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108370) pid(21583) version 11:1:1:4 [ CSSD]2013-05-23 18:04:39.045 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:04:39.045 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:04:39.045 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:04:39.045 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:39.045 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1124 (0x7f431bfe2210) [ CSSD]2013-05-23 18:04:39.045 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2210 (RPC#1124) tag(464002d) sent to node 1 [ CSSD]2013-05-23 18:04:39.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0464002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:39.045 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015659, gin 1, member 0 [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2210 (RPC#1124) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2210, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:04:39.046 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1124 [ CSSD]2013-05-23 18:04:39.052 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:04:39.052 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:39.052 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:39.052 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1125 (0x7f431bfe2290) [ CSSD]2013-05-23 18:04:39.052 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2290 (RPC#1125) tag(465002d) sent to node 1 [ CSSD]2013-05-23 18:04:39.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0465002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:39.053 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:39.053 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2290 (RPC#1125) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:39.053 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2290, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:04:39.053 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1125 [ CSSD]2013-05-23 18:04:39.053 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:04:39.255 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:04:39.255 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e6720) skgpid ospid 21583 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:04:39.288 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:39.976 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:04:39.977 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1126 (0x7f431bfe2310) [ CSSD]2013-05-23 18:04:39.977 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2310 (RPC#1126) tag(466002d) sent to node 1 [ CSSD]2013-05-23 18:04:39.977 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:39.977 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1126 (0x7f431bfe2310) [ CSSD]2013-05-23 18:04:39.977 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2310 (RPC#1126) state 6, flags 0x202 [ CSSD]2013-05-23 18:04:39.977 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2310, ret 0, client (nil) [ CSSD]2013-05-23 18:04:39.977 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1126 [ CSSD]2013-05-23 18:04:40.447 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:40.448 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:40.872 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:40.872 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:41.293 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:42.417 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:42.417 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015660, gin 1, member 0 [ CSSD]2013-05-23 18:04:42.417 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:04:42.417 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:04:42.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:42.424 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:04:42.424 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:04:42.878 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:42.878 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:43.318 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:44.885 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:44.885 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:45.331 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:45.451 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:45.451 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:46.278 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:04:46.279 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:04:46.279 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:04:46.279 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108370) pid(21676) version 11:1:1:4 [ CSSD]2013-05-23 18:04:46.286 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c121830) [ CSSD]2013-05-23 18:04:46.286 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 18:04:46.286 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121830 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:04:46.286 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:46.286 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1127 (0x7f431bfe2390) [ CSSD]2013-05-23 18:04:46.286 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2390 (RPC#1127) tag(467002d) sent to node 1 [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0467002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015661, gin 1, member 0 [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2390 (RPC#1127) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2390, ret 0, client 0x7f431c121830 member 0x7f431c048500 [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121830 [ CSSD]2013-05-23 18:04:46.287 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1127 [ CSSD]2013-05-23 18:04:46.293 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 18:04:46.293 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121830), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:46.293 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:46.293 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1128 (0x7f431bfe2410) [ CSSD]2013-05-23 18:04:46.294 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2410 (RPC#1128) tag(468002d) sent to node 1 [ CSSD]2013-05-23 18:04:46.294 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0468002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:46.294 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:46.294 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2410 (RPC#1128) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:46.294 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2410, ret 0, client 0x7f431c121830 member 0x7f431c048500 [ CSSD]2013-05-23 18:04:46.294 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1128 [ CSSD]2013-05-23 18:04:46.294 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:04:46.497 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:04:46.497 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e6720) skgpid ospid 21676 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:04:46.894 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:46.894 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:47.337 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:48.900 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:48.900 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:49.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:49.118 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015662, gin 1, member 0 [ CSSD]2013-05-23 18:04:49.118 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:04:49.118 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:04:49.126 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:49.126 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:04:49.126 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:04:49.388 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:50.000 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:04:50.000 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1129 (0x7f431bfe2490) [ CSSD]2013-05-23 18:04:50.000 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2490 (RPC#1129) tag(469002d) sent to node 1 [ CSSD]2013-05-23 18:04:50.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:50.001 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1129 (0x7f431bfe2490) [ CSSD]2013-05-23 18:04:50.001 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2490 (RPC#1129) state 6, flags 0x202 [ CSSD]2013-05-23 18:04:50.001 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2490, ret 0, client (nil) [ CSSD]2013-05-23 18:04:50.001 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1129 [ CSSD]2013-05-23 18:04:50.454 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:50.454 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:50.912 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:50.912 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:51.415 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:52.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:52.918 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:53.420 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:54.289 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:04:54.290 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:04:54.290 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:04:54.290 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c108370) pid(21740) version 11:1:1:4 [ CSSD]2013-05-23 18:04:54.297 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:04:54.297 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:04:54.297 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c135110, requested num -1 [ CSSD]2013-05-23 18:04:54.297 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:54.297 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1130 (0x7f431bfe2510) [ CSSD]2013-05-23 18:04:54.297 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2510 (RPC#1130) tag(46a002d) sent to node 1 [ CSSD]2013-05-23 18:04:54.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(046a002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:54.297 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015663, gin 1, member 0 [ CSSD]2013-05-23 18:04:54.297 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:04:54.298 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:04:54.298 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:04:54.298 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2510 (RPC#1130) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:54.298 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2510, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:04:54.298 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:04:54.298 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1130 [ CSSD]2013-05-23 18:04:54.304 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:04:54.304 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:54.304 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:04:54.304 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1131 (0x7f431bfe2590) [ CSSD]2013-05-23 18:04:54.304 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2590 (RPC#1131) tag(46b002d) sent to node 1 [ CSSD]2013-05-23 18:04:54.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(046b002d) incarnation(174631051) [ CSSD]2013-05-23 18:04:54.305 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:04:54.305 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2590 (RPC#1131) state 6, flags 0x100 [ CSSD]2013-05-23 18:04:54.305 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2590, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:04:54.305 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1131 [ CSSD]2013-05-23 18:04:54.305 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:04:54.507 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:04:54.507 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11ab50) skgpid ospid 21740 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:04:54.970 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:54.970 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:55.432 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:55.457 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:04:55.457 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:04:56.995 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:56.995 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:57.437 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:04:59.001 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:04:59.001 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:04:59.489 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:00.023 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:05:00.023 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1132 (0x7f431bfe2610) [ CSSD]2013-05-23 18:05:00.023 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2610 (RPC#1132) tag(46c002d) sent to node 1 [ CSSD]2013-05-23 18:05:00.024 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:00.024 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1132 (0x7f431bfe2610) [ CSSD]2013-05-23 18:05:00.024 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2610 (RPC#1132) state 6, flags 0x202 [ CSSD]2013-05-23 18:05:00.024 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2610, ret 0, client (nil) [ CSSD]2013-05-23 18:05:00.024 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1132 [ CSSD]2013-05-23 18:05:00.461 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:00.461 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:01.013 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:01.013 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:01.514 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:03.020 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:03.020 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:03.524 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:04.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:04.384 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015664, gin 1, member 0 [ CSSD]2013-05-23 18:05:04.384 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:05:04.384 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:05:04.392 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:04.392 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:05:04.392 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:05:05.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:05.070 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:05.464 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:05.464 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:05.530 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:07.076 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:07.076 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:07.537 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:08.432 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:05:08.433 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:05:08.434 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:05:08.434 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(21905) version 11:1:1:4 [ CSSD]2013-05-23 18:05:09.103 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:09.103 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:09.120 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:05:09.120 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid ospid 21905 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:09.539 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:05:09.540 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:05:09.541 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:05:09.541 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c108370) pid(22008) version 11:1:1:4 [ CSSD]2013-05-23 18:05:09.547 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108370), client(1/0x7f431c121830) [ CSSD]2013-05-23 18:05:09.547 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 18:05:09.547 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121830 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:05:09.547 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:09.547 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1133 (0x7f431bfe2690) [ CSSD]2013-05-23 18:05:09.547 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2690 (RPC#1133) tag(46d002d) sent to node 1 [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(046d002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015665, gin 1, member 0 [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2690 (RPC#1133) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2690, ret 0, client 0x7f431c121830 member 0x7f431c12f520 [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121830 [ CSSD]2013-05-23 18:05:09.548 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1133 [ CSSD]2013-05-23 18:05:09.555 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 18:05:09.555 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121830), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:09.555 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:09.555 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1134 (0x7f431bfe2710) [ CSSD]2013-05-23 18:05:09.555 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2710 (RPC#1134) tag(46e002d) sent to node 1 [ CSSD]2013-05-23 18:05:09.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(046e002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:09.555 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:09.555 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2710 (RPC#1134) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:09.555 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2710, ret 0, client 0x7f431c121830 member 0x7f431c12f520 [ CSSD]2013-05-23 18:05:09.556 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1134 [ CSSD]2013-05-23 18:05:09.556 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:05:09.558 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:09.758 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:05:09.758 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11ab50) skgpid ospid 22008 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:10.048 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:05:10.048 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1135 (0x7f431bfe2790) [ CSSD]2013-05-23 18:05:10.049 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2790 (RPC#1135) tag(46f002d) sent to node 1 [ CSSD]2013-05-23 18:05:10.049 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:10.049 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1135 (0x7f431bfe2790) [ CSSD]2013-05-23 18:05:10.049 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2790 (RPC#1135) state 6, flags 0x202 [ CSSD]2013-05-23 18:05:10.049 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2790, ret 0, client (nil) [ CSSD]2013-05-23 18:05:10.049 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1135 [ CSSD]2013-05-23 18:05:10.467 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:10.467 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:11.113 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:11.114 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:11.604 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:13.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:13.120 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:13.617 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:15.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:15.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:15.470 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:15.470 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:15.623 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:17.176 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:17.176 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:17.629 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:19.183 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:19.183 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:19.646 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:19.646 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015666, gin 1, member 0 [ CSSD]2013-05-23 18:05:19.646 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:05:19.646 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:05:19.652 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:19.652 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:05:19.652 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x22eec70) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:05:19.720 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:20.073 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:05:20.073 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1136 (0x7f431bfe2810) [ CSSD]2013-05-23 18:05:20.073 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2810 (RPC#1136) tag(470002d) sent to node 1 [ CSSD]2013-05-23 18:05:20.074 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:20.074 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1136 (0x7f431bfe2810) [ CSSD]2013-05-23 18:05:20.074 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2810 (RPC#1136) state 6, flags 0x202 [ CSSD]2013-05-23 18:05:20.074 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2810, ret 0, client (nil) [ CSSD]2013-05-23 18:05:20.074 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1136 [ CSSD]2013-05-23 18:05:20.473 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:20.473 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:20.917 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 2 (0x22d7cd0) [ CSSD]2013-05-23 18:05:20.917 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 2 (0x22d7cd0), grp DBHWPROD, member 1 [ CSSD]2013-05-23 18:05:20.917 [1191196992] >TRACE: clssgmUnregisterShared: client 2 (0x22d7cd0), grp DBHWPROD, member 1, share type 1 [ CSSD]2013-05-23 18:05:20.917 [1191196992] >TRACE: clssgmUnreferenceMember: global grock DBHWPROD member 1 refcount is 22 [ CSSD]2013-05-23 18:05:20.921 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x23113f0) [ CSSD]2013-05-23 18:05:20.921 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x23113f0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:05:20.921 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x23113f0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:05:20.921 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 18:05:20.921 [1191196992] >TRACE: clssgmDeadProc: proc 0x2310ec0 [ CSSD]2013-05-23 18:05:20.921 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x2310ec0) con(0x2327f70) skgpid 16806 ospid 16806 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:21.215 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:21.215 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:21.726 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:23.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:23.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:23.736 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:24.790 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:05:24.791 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:05:24.792 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:05:24.792 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108370) pid(22172) version 11:1:1:4 [ CSSD]2013-05-23 18:05:24.797 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:05:24.797 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:05:24.798 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c0e5200, requested num -1 [ CSSD]2013-05-23 18:05:24.798 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:24.798 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1137 (0x7f431bfe2890) [ CSSD]2013-05-23 18:05:24.798 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2890 (RPC#1137) tag(471002d) sent to node 1 [ CSSD]2013-05-23 18:05:24.798 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0471002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:24.798 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015667, gin 1, member 0 [ CSSD]2013-05-23 18:05:24.798 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:05:24.798 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:05:24.798 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:05:24.799 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2890 (RPC#1137) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:24.799 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2890, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 18:05:24.799 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:05:24.799 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1137 [ CSSD]2013-05-23 18:05:24.805 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:05:24.805 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:24.805 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:24.805 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1138 (0x7f431bfe2910) [ CSSD]2013-05-23 18:05:24.805 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2910 (RPC#1138) tag(472002d) sent to node 1 [ CSSD]2013-05-23 18:05:24.805 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0472002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:24.805 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:24.806 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2910 (RPC#1138) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:24.806 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2910, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 18:05:24.806 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1138 [ CSSD]2013-05-23 18:05:24.806 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:05:25.008 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:05:25.008 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e6720) skgpid ospid 22172 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:25.271 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:25.271 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:25.476 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:25.477 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:25.742 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:27.276 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:27.276 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:27.747 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:29.282 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:29.282 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:29.801 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:30.103 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:05:30.103 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1139 (0x7f431bfe2990) [ CSSD]2013-05-23 18:05:30.103 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2990 (RPC#1139) tag(473002d) sent to node 1 [ CSSD]2013-05-23 18:05:30.104 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:30.104 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1139 (0x7f431bfe2990) [ CSSD]2013-05-23 18:05:30.104 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2990 (RPC#1139) state 6, flags 0x202 [ CSSD]2013-05-23 18:05:30.104 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2990, ret 0, client (nil) [ CSSD]2013-05-23 18:05:30.104 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1139 [ CSSD]2013-05-23 18:05:30.479 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:30.480 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:31.293 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:31.293 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:31.827 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:33.299 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:33.299 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:33.837 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:34.916 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:34.916 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015668, gin 1, member 0 [ CSSD]2013-05-23 18:05:34.916 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:05:34.916 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:05:34.923 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:34.923 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:05:34.923 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:05:35.371 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:35.371 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:35.482 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:35.483 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:35.843 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:37.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:37.418 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:37.868 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:39.424 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:39.424 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:39.901 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:40.040 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:05:40.041 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:05:40.041 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:05:40.041 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108370) pid(22311) version 11:1:1:4 [ CSSD]2013-05-23 18:05:40.047 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:40.047 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:40.047 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c11ab50, requested num -1 [ CSSD]2013-05-23 18:05:40.047 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:40.047 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1140 (0x7f431bfe2a10) [ CSSD]2013-05-23 18:05:40.047 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2a10 (RPC#1140) tag(474002d) sent to node 1 [ CSSD]2013-05-23 18:05:40.047 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0474002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:40.047 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015669, gin 1, member 0 [ CSSD]2013-05-23 18:05:40.047 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:05:40.048 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:05:40.048 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:05:40.048 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2a10 (RPC#1140) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:40.048 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2a10, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:05:40.048 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:05:40.048 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1140 [ CSSD]2013-05-23 18:05:40.054 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:40.055 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:40.055 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:40.055 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1141 (0x7f431bfe2a90) [ CSSD]2013-05-23 18:05:40.055 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2a90 (RPC#1141) tag(475002d) sent to node 1 [ CSSD]2013-05-23 18:05:40.055 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0475002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:40.055 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:40.055 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2a90 (RPC#1141) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:40.055 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2a90, ret 0, client 0x7f431c0e6e30 member 0x7f431c048500 [ CSSD]2013-05-23 18:05:40.055 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1141 [ CSSD]2013-05-23 18:05:40.055 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:05:40.134 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:05:40.134 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1142 (0x7f431bfe2b10) [ CSSD]2013-05-23 18:05:40.135 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2b10 (RPC#1142) tag(476002d) sent to node 1 [ CSSD]2013-05-23 18:05:40.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:40.135 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1142 (0x7f431bfe2b10) [ CSSD]2013-05-23 18:05:40.135 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2b10 (RPC#1142) state 6, flags 0x202 [ CSSD]2013-05-23 18:05:40.135 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2b10, ret 0, client (nil) [ CSSD]2013-05-23 18:05:40.135 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1142 [ CSSD]2013-05-23 18:05:40.258 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:05:40.258 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e6720) skgpid ospid 22311 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:40.485 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:40.486 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:41.475 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:41.475 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:41.926 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:42.678 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:42.678 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015670, gin 1, member 0 [ CSSD]2013-05-23 18:05:42.678 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:05:42.678 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:05:42.685 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:42.685 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:05:42.685 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:05:43.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:43.481 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:43.938 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:45.489 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:45.489 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:45.492 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:45.492 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:45.943 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:46.539 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:05:46.540 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:05:46.540 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:05:46.540 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c135110) proc(0x7f431c108370) pid(22417) version 11:1:1:4 [ CSSD]2013-05-23 18:05:46.546 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:46.546 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:46.546 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c0e6720, requested num -1 [ CSSD]2013-05-23 18:05:46.546 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:46.547 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1143 (0x7f431bfe2b90) [ CSSD]2013-05-23 18:05:46.547 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2b90 (RPC#1143) tag(477002d) sent to node 1 [ CSSD]2013-05-23 18:05:46.547 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0477002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:46.547 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015671, gin 1, member 0 [ CSSD]2013-05-23 18:05:46.547 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:05:46.548 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:05:46.548 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:05:46.548 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2b90 (RPC#1143) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:46.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2b90, ret 0, client 0x7f431c0e6e30 member 0x7f431c12f520 [ CSSD]2013-05-23 18:05:46.548 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:05:46.548 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1143 [ CSSD]2013-05-23 18:05:46.555 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:46.555 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:46.555 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:46.555 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1144 (0x7f431bfe2c10) [ CSSD]2013-05-23 18:05:46.555 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2c10 (RPC#1144) tag(478002d) sent to node 1 [ CSSD]2013-05-23 18:05:46.555 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0478002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:46.555 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:46.555 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2c10 (RPC#1144) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:46.555 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2c10, ret 0, client 0x7f431c0e6e30 member 0x7f431c12f520 [ CSSD]2013-05-23 18:05:46.555 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1144 [ CSSD]2013-05-23 18:05:46.555 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:05:46.758 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:05:46.758 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c135110) skgpid ospid 22417 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:47.498 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:47.498 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:47.967 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:49.525 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:49.525 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:50.001 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:50.172 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:50.172 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015672, gin 1, member 0 [ CSSD]2013-05-23 18:05:50.172 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:05:50.172 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:05:50.174 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:05:50.174 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1145 (0x7f431bfe2c90) [ CSSD]2013-05-23 18:05:50.174 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2c90 (RPC#1145) tag(479002d) sent to node 1 [ CSSD]2013-05-23 18:05:50.174 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:50.175 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1145 (0x7f431bfe2c90) [ CSSD]2013-05-23 18:05:50.175 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2c90 (RPC#1145) state 6, flags 0x202 [ CSSD]2013-05-23 18:05:50.175 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2c90, ret 0, client (nil) [ CSSD]2013-05-23 18:05:50.175 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1145 [ CSSD]2013-05-23 18:05:50.214 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:50.214 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:05:50.214 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:05:50.492 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:50.492 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:51.576 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:51.576 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:52.026 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:53.602 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:53.602 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:54.037 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:54.495 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:54.495 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:05:55.290 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:05:55.291 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:05:55.291 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:05:55.291 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c108370) pid(22500) version 11:1:1:4 [ CSSD]2013-05-23 18:05:55.297 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108370), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:55.297 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:55.297 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c135110, requested num -1 [ CSSD]2013-05-23 18:05:55.297 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:55.297 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1146 (0x7f431bfe2d10) [ CSSD]2013-05-23 18:05:55.297 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2d10 (RPC#1146) tag(47a002d) sent to node 1 [ CSSD]2013-05-23 18:05:55.297 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(047a002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:55.297 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015673, gin 1, member 0 [ CSSD]2013-05-23 18:05:55.297 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c13edd0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:05:55.298 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:05:55.298 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:05:55.298 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2d10 (RPC#1146) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:55.298 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2d10, ret 0, client 0x7f431c0e6e30 member 0x7f431c12f520 [ CSSD]2013-05-23 18:05:55.298 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:05:55.298 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1146 [ CSSD]2013-05-23 18:05:55.304 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:05:55.304 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:55.304 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c12f520) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:05:55.304 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1147 (0x7f431bfe2d90) [ CSSD]2013-05-23 18:05:55.304 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2d90 (RPC#1147) tag(47b002d) sent to node 1 [ CSSD]2013-05-23 18:05:55.305 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(047b002d) incarnation(174631051) [ CSSD]2013-05-23 18:05:55.305 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:05:55.305 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2d90 (RPC#1147) state 6, flags 0x100 [ CSSD]2013-05-23 18:05:55.305 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2d90, ret 0, client 0x7f431c0e6e30 member 0x7f431c12f520 [ CSSD]2013-05-23 18:05:55.305 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1147 [ CSSD]2013-05-23 18:05:55.305 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c12f520) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:05:55.507 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:05:55.507 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c11ab50) skgpid ospid 22500 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:05:55.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:55.613 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:56.043 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:57.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:57.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:05:58.049 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:05:59.498 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:05:59.498 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:05:59.626 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:05:59.626 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:00.102 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:00.198 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:06:00.199 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1148 (0x7f431bfe2e10) [ CSSD]2013-05-23 18:06:00.199 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2e10 (RPC#1148) tag(47c002d) sent to node 1 [ CSSD]2013-05-23 18:06:00.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:00.200 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1148 (0x7f431bfe2e10) [ CSSD]2013-05-23 18:06:00.200 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2e10 (RPC#1148) state 6, flags 0x202 [ CSSD]2013-05-23 18:06:00.200 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2e10, ret 0, client (nil) [ CSSD]2013-05-23 18:06:00.200 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1148 [ CSSD]2013-05-23 18:06:01.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:01.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:02.108 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:03.643 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:03.643 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:04.120 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:04.501 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:04.501 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:05.434 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:05.434 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015674, gin 1, member 0 [ CSSD]2013-05-23 18:06:05.434 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:06:05.434 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:06:05.442 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:05.442 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:06:05.442 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:06:05.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:05.694 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:06.144 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:07.719 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:07.719 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:08.150 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:09.504 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:09.504 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:09.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:09.726 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:10.202 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:10.231 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:06:10.231 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1149 (0x7f431bfe2e90) [ CSSD]2013-05-23 18:06:10.231 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2e90 (RPC#1149) tag(47d002d) sent to node 1 [ CSSD]2013-05-23 18:06:10.232 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:10.232 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1149 (0x7f431bfe2e90) [ CSSD]2013-05-23 18:06:10.232 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2e90 (RPC#1149) state 6, flags 0x202 [ CSSD]2013-05-23 18:06:10.232 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe2e90, ret 0, client (nil) [ CSSD]2013-05-23 18:06:10.232 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1149 [ CSSD]2013-05-23 18:06:10.540 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:06:10.541 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:06:10.542 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:06:10.542 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c108370) pid(22666) version 11:1:1:4 [ CSSD]2013-05-23 18:06:10.549 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108370), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:06:10.549 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:06:10.549 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c135110, requested num -1 [ CSSD]2013-05-23 18:06:10.549 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:10.549 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1150 (0x7f431bfe2f10) [ CSSD]2013-05-23 18:06:10.549 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2f10 (RPC#1150) tag(47e002d) sent to node 1 [ CSSD]2013-05-23 18:06:10.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(047e002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:10.549 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015675, gin 1, member 0 [ CSSD]2013-05-23 18:06:10.549 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0958d0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:06:10.550 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:06:10.550 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:06:10.550 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2f10 (RPC#1150) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:10.550 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe2f10, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 18:06:10.550 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:06:10.550 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1150 [ CSSD]2013-05-23 18:06:10.557 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:06:10.557 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:10.557 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c048500) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:10.557 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1151 (0x7f431bfe2f90) [ CSSD]2013-05-23 18:06:10.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe2f90 (RPC#1151) tag(47f002d) sent to node 1 [ CSSD]2013-05-23 18:06:10.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(047f002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:10.557 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:10.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe2f90 (RPC#1151) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:10.557 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe2f90, ret 0, client 0x7f431c0cf720 member 0x7f431c048500 [ CSSD]2013-05-23 18:06:10.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1151 [ CSSD]2013-05-23 18:06:10.558 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c048500) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:06:10.760 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:06:10.760 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e6720) skgpid ospid 22666 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:11.737 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:11.737 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:12.208 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:13.744 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:13.744 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:14.220 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:14.507 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:14.507 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:15.795 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:15.795 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:16.245 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:17.801 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:17.801 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:18.250 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:19.510 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:19.510 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:19.808 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:19.808 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:20.264 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:06:20.265 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1152 (0x7f431bfe3010) [ CSSD]2013-05-23 18:06:20.265 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3010 (RPC#1152) tag(480002d) sent to node 1 [ CSSD]2013-05-23 18:06:20.265 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:20.265 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1152 (0x7f431bfe3010) [ CSSD]2013-05-23 18:06:20.265 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3010 (RPC#1152) state 6, flags 0x202 [ CSSD]2013-05-23 18:06:20.265 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3010, ret 0, client (nil) [ CSSD]2013-05-23 18:06:20.265 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1152 [ CSSD]2013-05-23 18:06:20.292 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108370) allocated [ CSSD]2013-05-23 18:06:20.293 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:06:20.293 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108370 - 1,2,3 [ CSSD]2013-05-23 18:06:20.293 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e5200) proc(0x7f431c108370) pid(21186/21186) version 11:1:1:4 [ CSSD]2013-05-23 18:06:20.303 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:20.309 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c108370), client(1/0x7f431c121830) [ CSSD]2013-05-23 18:06:20.309 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 18:06:20.309 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:06:20.309 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 18:06:20.686 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:20.686 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015676, gin 1, member 0 [ CSSD]2013-05-23 18:06:20.686 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:06:20.686 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:06:20.726 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:20.726 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:06:20.726 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:06:21.838 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:21.838 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:22.309 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:23.651 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c13edd0) allocated [ CSSD]2013-05-23 18:06:23.652 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:06:23.652 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c13edd0 - 1,2,3 [ CSSD]2013-05-23 18:06:23.652 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c13edd0) pid(22814) version 11:1:1:4 [ CSSD]2013-05-23 18:06:23.659 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c13edd0), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:06:23.659 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:06:23.659 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x7f431c0cf720 with con 0x7f431c124850, requested num -1 [ CSSD]2013-05-23 18:06:23.659 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 18:06:23.659 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1153 (0x7f431bfe3090) [ CSSD]2013-05-23 18:06:23.659 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3090 (RPC#1153) tag(481002d) sent to node 1 [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0481002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015677, gin 1, member 0 [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c108780/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3090 (RPC#1153) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3090, ret 0, client 0x7f431c0cf720 member 0x7f431c108a80 [ CSSD]2013-05-23 18:06:23.660 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:06:23.661 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1153 [ CSSD]2013-05-23 18:06:23.667 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:06:23.667 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 18:06:23.667 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c108a80) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 18:06:23.667 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1154 (0x7f431bfe3110) [ CSSD]2013-05-23 18:06:23.667 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3110 (RPC#1154) tag(482002d) sent to node 1 [ CSSD]2013-05-23 18:06:23.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0482002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:23.668 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 18:06:23.668 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3110 (RPC#1154) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:23.668 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3110, ret 0, client 0x7f431c0cf720 member 0x7f431c108a80 [ CSSD]2013-05-23 18:06:23.668 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1154 [ CSSD]2013-05-23 18:06:23.668 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x7f431c108a80) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:06:23.779 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1348f0) allocated [ CSSD]2013-05-23 18:06:23.781 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:06:23.781 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1348f0 - 1,2,3 [ CSSD]2013-05-23 18:06:23.781 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0e6720) proc(0x7f431c1348f0) pid(22819) version 11:1:1:4 [ CSSD]2013-05-23 18:06:23.796 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:06:23.797 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:06:23.800 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1348f0 [ CSSD]2013-05-23 18:06:23.800 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1348f0) con(0x7f431c0e6720) skgpid ospid 22819 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:23.864 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:23.864 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:23.870 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c13edd0 [ CSSD]2013-05-23 18:06:23.870 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c13edd0) con(0x7f431c11ab50) skgpid ospid 22814 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:24.320 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:24.513 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:24.513 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:25.485 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x22f7ce0) [ CSSD]2013-05-23 18:06:25.485 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x22f7ce0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:06:25.485 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x22f7ce0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:06:25.485 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 18:06:25.485 [1191196992] >TRACE: clssgmDeadProc: proc 0x22ff670 [ CSSD]2013-05-23 18:06:25.485 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22ff670) con(0x23037d0) skgpid 27626 ospid 27626 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:25.792 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 18:06:25.793 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:06:25.794 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 18:06:25.794 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c12b920) pid(22840) version 11:1:1:4 [ CSSD]2013-05-23 18:06:25.799 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c12b920), client(1/0x7f431c0cf720) [ CSSD]2013-05-23 18:06:25.799 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:06:25.799 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0cf720 with con 0x7f431c1347e0, requested num -1 [ CSSD]2013-05-23 18:06:25.799 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:25.799 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1155 (0x7f431bfe3190) [ CSSD]2013-05-23 18:06:25.799 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3190 (RPC#1155) tag(483002d) sent to node 1 [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0483002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015678, gin 1, member 0 [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c1086b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3190 (RPC#1155) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3190, ret 0, client 0x7f431c0cf720 member 0x7f431c1247c0 [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0cf720 [ CSSD]2013-05-23 18:06:25.800 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1155 [ CSSD]2013-05-23 18:06:25.807 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0cf720) [ CSSD]2013-05-23 18:06:25.807 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0cf720), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:25.807 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1247c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:25.807 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1156 (0x7f431bfe3210) [ CSSD]2013-05-23 18:06:25.807 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3210 (RPC#1156) tag(484002d) sent to node 1 [ CSSD]2013-05-23 18:06:25.807 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0484002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:25.807 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:25.807 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3210 (RPC#1156) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:25.807 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3210, ret 0, client 0x7f431c0cf720 member 0x7f431c1247c0 [ CSSD]2013-05-23 18:06:25.808 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1156 [ CSSD]2013-05-23 18:06:25.808 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1247c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:06:25.895 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:25.895 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:26.010 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c12b920 [ CSSD]2013-05-23 18:06:26.010 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c12b920) con(0x7f431c109f70) skgpid ospid 22840 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:26.325 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:27.902 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:27.902 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:28.349 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:29.516 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:29.516 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:29.908 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:29.908 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:30.297 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:06:30.297 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1157 (0x7f431bfe3290) [ CSSD]2013-05-23 18:06:30.298 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3290 (RPC#1157) tag(485002d) sent to node 1 [ CSSD]2013-05-23 18:06:30.298 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:30.298 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1157 (0x7f431bfe3290) [ CSSD]2013-05-23 18:06:30.298 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3290 (RPC#1157) state 6, flags 0x202 [ CSSD]2013-05-23 18:06:30.298 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3290, ret 0, client (nil) [ CSSD]2013-05-23 18:06:30.298 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1157 [ CSSD]2013-05-23 18:06:30.352 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:31.918 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:31.918 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:32.377 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:33.925 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:33.925 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:34.451 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:34.519 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:34.520 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:35.935 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:35.935 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:35.939 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:35.939 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015679, gin 1, member 0 [ CSSD]2013-05-23 18:06:35.939 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:06:35.939 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:06:35.946 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:35.946 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:06:35.946 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:06:36.456 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:37.212 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c12b920) allocated [ CSSD]2013-05-23 18:06:37.213 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:06:37.214 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c12b920 - 1,2,3 [ CSSD]2013-05-23 18:06:37.214 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c11ab50) proc(0x7f431c12b920) pid(22146/22146) version 11:1:1:4 [ CSSD]2013-05-23 18:06:37.230 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c12b920), client(1/0x7f431c10a560) [ CSSD]2013-05-23 18:06:37.230 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c10a560) [ CSSD]2013-05-23 18:06:37.230 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:06:37.230 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 18:06:37.981 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:37.981 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:38.509 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:38.522 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:38.522 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:06:39.993 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:39.993 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:40.323 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:06:40.323 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1158 (0x7f431bfe3310) [ CSSD]2013-05-23 18:06:40.323 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3310 (RPC#1158) tag(486002d) sent to node 1 [ CSSD]2013-05-23 18:06:40.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:40.324 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1158 (0x7f431bfe3310) [ CSSD]2013-05-23 18:06:40.324 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3310 (RPC#1158) state 6, flags 0x202 [ CSSD]2013-05-23 18:06:40.324 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3310, ret 0, client (nil) [ CSSD]2013-05-23 18:06:40.324 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1158 [ CSSD]2013-05-23 18:06:40.515 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:41.042 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134cc0) allocated [ CSSD]2013-05-23 18:06:41.043 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:06:41.043 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134cc0 - 1,2,3 [ CSSD]2013-05-23 18:06:41.043 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1347e0) proc(0x7f431c134cc0) pid(22996) version 11:1:1:4 [ CSSD]2013-05-23 18:06:41.049 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134cc0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:41.049 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:41.049 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c134df0, requested num -1 [ CSSD]2013-05-23 18:06:41.049 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:41.049 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1159 (0x7f431bfe3390) [ CSSD]2013-05-23 18:06:41.050 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3390 (RPC#1159) tag(487002d) sent to node 1 [ CSSD]2013-05-23 18:06:41.050 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0487002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:41.050 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015680, gin 1, member 0 [ CSSD]2013-05-23 18:06:41.050 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ad0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:06:41.050 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:06:41.050 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:06:41.050 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3390 (RPC#1159) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:41.051 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3390, ret 0, client 0x7f431c0e6e30 member 0x7f431c134ad0 [ CSSD]2013-05-23 18:06:41.051 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:06:41.051 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1159 [ CSSD]2013-05-23 18:06:41.057 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:41.058 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:41.058 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134ad0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:41.058 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1160 (0x7f431bfe3410) [ CSSD]2013-05-23 18:06:41.058 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3410 (RPC#1160) tag(488002d) sent to node 1 [ CSSD]2013-05-23 18:06:41.058 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0488002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:41.058 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:41.058 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3410 (RPC#1160) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:41.058 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3410, ret 0, client 0x7f431c0e6e30 member 0x7f431c134ad0 [ CSSD]2013-05-23 18:06:41.058 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1160 [ CSSD]2013-05-23 18:06:41.058 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134ad0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:06:41.261 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134cc0 [ CSSD]2013-05-23 18:06:41.261 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134cc0) con(0x7f431c1347e0) skgpid ospid 22996 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:41.999 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:41.999 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:42.521 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:42.941 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:42.941 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015681, gin 1, member 0 [ CSSD]2013-05-23 18:06:42.941 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:06:42.941 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:06:42.949 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:42.949 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:06:42.949 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:06:43.525 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:43.526 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:44.006 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:44.006 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:44.532 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:46.015 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:46.015 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:46.557 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:46.793 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129640) allocated [ CSSD]2013-05-23 18:06:46.794 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:06:46.795 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129640 - 1,2,3 [ CSSD]2013-05-23 18:06:46.795 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095990) proc(0x7f431c129640) pid(23081) version 11:1:1:4 [ CSSD]2013-05-23 18:06:46.801 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129640), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:46.801 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:46.801 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c134d10, requested num -1 [ CSSD]2013-05-23 18:06:46.801 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:46.801 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1161 (0x7f431bfe3490) [ CSSD]2013-05-23 18:06:46.802 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3490 (RPC#1161) tag(489002d) sent to node 1 [ CSSD]2013-05-23 18:06:46.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0489002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:46.802 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015682, gin 1, member 0 [ CSSD]2013-05-23 18:06:46.802 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ba0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:06:46.802 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:06:46.802 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:06:46.802 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3490 (RPC#1161) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:46.803 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3490, ret 0, client 0x7f431c0e6e30 member 0x7f431c120ea0 [ CSSD]2013-05-23 18:06:46.803 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:06:46.803 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1161 [ CSSD]2013-05-23 18:06:46.811 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:46.811 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:46.811 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c120ea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:46.811 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1162 (0x7f431bfe3510) [ CSSD]2013-05-23 18:06:46.811 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3510 (RPC#1162) tag(48a002d) sent to node 1 [ CSSD]2013-05-23 18:06:46.811 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(048a002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:46.811 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:46.811 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3510 (RPC#1162) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:46.811 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3510, ret 0, client 0x7f431c0e6e30 member 0x7f431c120ea0 [ CSSD]2013-05-23 18:06:46.812 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1162 [ CSSD]2013-05-23 18:06:46.812 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c120ea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:06:47.014 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129640 [ CSSD]2013-05-23 18:06:47.014 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129640) con(0x7f431c095990) skgpid ospid 23081 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:48.022 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:48.022 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:48.529 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:48.529 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:48.609 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:50.093 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:50.093 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:50.348 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:06:50.348 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1163 (0x7f431bfe3590) [ CSSD]2013-05-23 18:06:50.348 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3590 (RPC#1163) tag(48b002d) sent to node 1 [ CSSD]2013-05-23 18:06:50.349 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:50.349 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1163 (0x7f431bfe3590) [ CSSD]2013-05-23 18:06:50.349 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3590 (RPC#1163) state 6, flags 0x202 [ CSSD]2013-05-23 18:06:50.349 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3590, ret 0, client (nil) [ CSSD]2013-05-23 18:06:50.349 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1163 [ CSSD]2013-05-23 18:06:50.615 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:51.193 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:51.193 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015683, gin 1, member 0 [ CSSD]2013-05-23 18:06:51.193 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:06:51.193 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:06:51.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:51.200 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:06:51.200 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:06:52.100 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:52.100 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:52.621 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:53.532 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:53.532 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:06:54.107 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:54.107 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:54.631 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:56.117 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:56.117 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:56.296 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 18:06:56.337 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:06:56.338 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 18:06:56.338 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c134ba0) pid(23159) version 11:1:1:4 [ CSSD]2013-05-23 18:06:56.354 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:56.354 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:56.354 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c1347e0, requested num -1 [ CSSD]2013-05-23 18:06:56.354 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:56.354 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1164 (0x7f431bfe3610) [ CSSD]2013-05-23 18:06:56.354 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3610 (RPC#1164) tag(48c002d) sent to node 1 [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(048c002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015684, gin 1, member 0 [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ad0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3610 (RPC#1164) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3610, ret 0, client 0x7f431c0e6e30 member 0x7f431c1349b0 [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:06:56.355 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1164 [ CSSD]2013-05-23 18:06:56.361 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:06:56.362 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:56.362 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c1349b0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:06:56.362 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1165 (0x7f431bfe3690) [ CSSD]2013-05-23 18:06:56.362 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3690 (RPC#1165) tag(48d002d) sent to node 1 [ CSSD]2013-05-23 18:06:56.362 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(048d002d) incarnation(174631051) [ CSSD]2013-05-23 18:06:56.362 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:06:56.362 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3690 (RPC#1165) state 6, flags 0x100 [ CSSD]2013-05-23 18:06:56.362 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3690, ret 0, client 0x7f431c0e6e30 member 0x7f431c1349b0 [ CSSD]2013-05-23 18:06:56.362 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1165 [ CSSD]2013-05-23 18:06:56.362 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c1349b0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:06:56.565 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 18:06:56.565 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c109f70) skgpid ospid 23159 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:06:56.656 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:06:57.535 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:06:57.535 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:06:58.142 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:06:58.142 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:06:58.710 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:00.194 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:00.194 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:00.384 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:07:00.384 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1166 (0x7f431bfe3710) [ CSSD]2013-05-23 18:07:00.384 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3710 (RPC#1166) tag(48e002d) sent to node 1 [ CSSD]2013-05-23 18:07:00.385 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:00.385 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1166 (0x7f431bfe3710) [ CSSD]2013-05-23 18:07:00.385 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3710 (RPC#1166) state 6, flags 0x202 [ CSSD]2013-05-23 18:07:00.385 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3710, ret 0, client (nil) [ CSSD]2013-05-23 18:07:00.385 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1166 [ CSSD]2013-05-23 18:07:00.716 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:02.200 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:02.200 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:02.538 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:02.538 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:02.722 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:04.207 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:04.207 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:04.773 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:06.216 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:06.217 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:06.458 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:06.458 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015685, gin 1, member 0 [ CSSD]2013-05-23 18:07:06.458 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:06.458 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:06.466 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:06.466 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:06.466 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:06.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:07.542 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:07.542 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:08.223 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:08.223 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:08.810 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:10.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:10.234 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:10.408 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:07:10.408 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1167 (0x7f431bfe3790) [ CSSD]2013-05-23 18:07:10.408 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3790 (RPC#1167) tag(48f002d) sent to node 1 [ CSSD]2013-05-23 18:07:10.409 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:10.409 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1167 (0x7f431bfe3790) [ CSSD]2013-05-23 18:07:10.409 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3790 (RPC#1167) state 6, flags 0x202 [ CSSD]2013-05-23 18:07:10.409 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3790, ret 0, client (nil) [ CSSD]2013-05-23 18:07:10.409 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1167 [ CSSD]2013-05-23 18:07:10.816 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:11.598 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c129640) allocated [ CSSD]2013-05-23 18:07:11.598 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:07:11.599 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c129640 - 1,2,3 [ CSSD]2013-05-23 18:07:11.599 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095990) proc(0x7f431c129640) pid(23305) version 11:1:1:4 [ CSSD]2013-05-23 18:07:11.605 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c129640), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:11.605 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:11.605 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c134d10, requested num -1 [ CSSD]2013-05-23 18:07:11.605 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:11.605 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1168 (0x7f431bfe3810) [ CSSD]2013-05-23 18:07:11.605 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3810 (RPC#1168) tag(490002d) sent to node 1 [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0490002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015686, gin 1, member 0 [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ba0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3810 (RPC#1168) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3810, ret 0, client 0x7f431c0e6e30 member 0x7f431c120ea0 [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:07:11.606 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1168 [ CSSD]2013-05-23 18:07:11.612 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:11.612 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:11.612 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c120ea0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:11.612 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1169 (0x7f431bfe3890) [ CSSD]2013-05-23 18:07:11.612 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3890 (RPC#1169) tag(491002d) sent to node 1 [ CSSD]2013-05-23 18:07:11.612 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0491002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:11.612 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:11.612 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3890 (RPC#1169) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:11.612 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3890, ret 0, client 0x7f431c0e6e30 member 0x7f431c120ea0 [ CSSD]2013-05-23 18:07:11.613 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1169 [ CSSD]2013-05-23 18:07:11.613 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c120ea0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:07:11.815 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c129640 [ CSSD]2013-05-23 18:07:11.815 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c129640) con(0x7f431c095990) skgpid ospid 23305 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:12.239 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:12.239 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:12.545 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:12.545 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:12.822 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:14.245 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:14.245 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:14.833 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:15.673 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:15.673 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015687, gin 1, member 0 [ CSSD]2013-05-23 18:07:15.673 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:15.673 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:15.680 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:15.680 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:15.680 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:16.298 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:16.298 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:16.547 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:16.547 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:07:16.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:18.304 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:18.304 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:18.911 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:20.314 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:20.314 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:20.431 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:07:20.432 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1170 (0x7f431bfe3910) [ CSSD]2013-05-23 18:07:20.432 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3910 (RPC#1170) tag(492002d) sent to node 1 [ CSSD]2013-05-23 18:07:20.432 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:20.432 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1170 (0x7f431bfe3910) [ CSSD]2013-05-23 18:07:20.432 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3910 (RPC#1170) state 6, flags 0x202 [ CSSD]2013-05-23 18:07:20.432 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3910, ret 0, client (nil) [ CSSD]2013-05-23 18:07:20.433 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1170 [ CSSD]2013-05-23 18:07:20.917 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:21.550 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:21.551 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:21.717 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:21.717 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015688, gin 1, member 0 [ CSSD]2013-05-23 18:07:21.717 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:21.717 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:21.724 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:21.724 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:21.725 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:22.341 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:22.341 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:22.417 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 18:07:22.417 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:07:22.418 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 18:07:22.418 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c134ba0) pid(23620) version 11:1:1:4 [ CSSD]2013-05-23 18:07:22.552 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c1213f0) allocated [ CSSD]2013-05-23 18:07:22.553 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:07:22.554 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c1213f0 - 1,2,3 [ CSSD]2013-05-23 18:07:22.554 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c134aa0) proc(0x7f431c1213f0) pid(23630) version 11:1:1:4 [ CSSD]2013-05-23 18:07:22.570 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:07:22.572 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:07:22.574 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c1213f0 [ CSSD]2013-05-23 18:07:22.574 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c1213f0) con(0x7f431c134aa0) skgpid ospid 23630 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:22.646 [1191196992] >TRACE: clssgmRegisterClient: proc(17/0x22caa20), client(88/0x7f431c13f030) [ CSSD]2013-05-23 18:07:22.646 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 88 (0x7f431c13f030) [ CSSD]2013-05-23 18:07:22.646 [1191196992] >TRACE: clssgmJoinGrock: global grock ORA_CLSRCSN_SRV_HWPROD2 new client 0x7f431c13f030 with con 0x7f431c1297c0, requested num -1 [ CSSD]2013-05-23 18:07:22.647 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 18:07:22.647 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1171 (0x7f431bfe3990) [ CSSD]2013-05-23 18:07:22.647 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3990 (RPC#1171) tag(493002d) sent to node 1 [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0493002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock ORA_CLSRCSN_SRV_HWPROD2, gid 2015689, gin 1, member 0 [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(2) grock (0x7f431c120ad0/ORA_CLSRCSN_SRV_HWPROD2) [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(ORA_CLSRCSN_SRV_HWPROD2) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0) [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD2 member(0/Local) node(2) flags 0x2 0x1000002 [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3990 (RPC#1171) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3990, ret 0, client 0x7f431c13f030 member 0x7f431c129640 [ CSSD]2013-05-23 18:07:22.647 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c13f030 [ CSSD]2013-05-23 18:07:22.648 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1171 [ CSSD]2013-05-23 18:07:22.653 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 88 (0x7f431c13f030) [ CSSD]2013-05-23 18:07:22.653 [1191196992] >TRACE: clssgmExitGrock: client 88 (0x7f431c13f030), grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 18:07:22.653 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c129640) in global grock ORA_CLSRCSN_SRV_HWPROD2 [ CSSD]2013-05-23 18:07:22.653 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1172 (0x7f431bfe3a10) [ CSSD]2013-05-23 18:07:22.654 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3a10 (RPC#1172) tag(494002d) sent to node 1 [ CSSD]2013-05-23 18:07:22.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0494002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:22.654 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD2, member 0 [ CSSD]2013-05-23 18:07:22.654 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3a10 (RPC#1172) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:22.654 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3a10, ret 0, client 0x7f431c13f030 member 0x7f431c129640 [ CSSD]2013-05-23 18:07:22.654 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1172 [ CSSD]2013-05-23 18:07:22.654 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD2, member number 0 (0x7f431c129640) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:07:22.655 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 18:07:22.655 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c109f70) skgpid ospid 23620 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:22.923 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:24.348 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:24.348 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:24.934 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:26.347 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:26.347 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:26.553 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:26.553 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:26.847 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 18:07:26.849 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:07:26.849 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 18:07:26.849 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c134ba0) pid(23679) version 11:1:1:4 [ CSSD]2013-05-23 18:07:26.855 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:26.855 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:26.855 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c0e6e30 with con 0x7f431c134dc0, requested num -1 [ CSSD]2013-05-23 18:07:26.855 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:26.855 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1173 (0x7f431bfe3a90) [ CSSD]2013-05-23 18:07:26.855 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3a90 (RPC#1173) tag(495002d) sent to node 1 [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0495002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015690, gin 1, member 0 [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c120ad0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3a90 (RPC#1173) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3a90, ret 0, client 0x7f431c0e6e30 member 0x7f431c134aa0 [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c0e6e30 [ CSSD]2013-05-23 18:07:26.856 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1173 [ CSSD]2013-05-23 18:07:26.862 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:26.862 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c0e6e30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:26.862 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c134aa0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:26.862 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1174 (0x7f431bfe3b10) [ CSSD]2013-05-23 18:07:26.862 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3b10 (RPC#1174) tag(496002d) sent to node 1 [ CSSD]2013-05-23 18:07:26.863 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(0496002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:26.863 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:26.863 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3b10 (RPC#1174) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:26.863 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3b10, ret 0, client 0x7f431c0e6e30 member 0x7f431c134aa0 [ CSSD]2013-05-23 18:07:26.863 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1174 [ CSSD]2013-05-23 18:07:26.863 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c134aa0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:07:26.960 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:27.066 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c134ba0 [ CSSD]2013-05-23 18:07:27.066 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c134ba0) con(0x7f431c109f70) skgpid ospid 23679 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:28.353 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:28.353 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:29.012 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:30.406 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:30.406 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:30.471 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:07:30.471 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1175 (0x7f431bfe3b90) [ CSSD]2013-05-23 18:07:30.471 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3b90 (RPC#1175) tag(497002d) sent to node 1 [ CSSD]2013-05-23 18:07:30.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:30.472 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1175 (0x7f431bfe3b90) [ CSSD]2013-05-23 18:07:30.472 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3b90 (RPC#1175) state 6, flags 0x202 [ CSSD]2013-05-23 18:07:30.472 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3b90, ret 0, client (nil) [ CSSD]2013-05-23 18:07:30.472 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1175 [ CSSD]2013-05-23 18:07:31.017 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:31.556 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:31.556 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:32.412 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:32.412 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:33.023 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:34.438 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:34.438 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:35.034 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:35.559 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:35.559 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:07:36.184 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c134ba0) allocated [ CSSD]2013-05-23 18:07:36.185 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:07:36.185 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c134ba0 - 1,2,3 [ CSSD]2013-05-23 18:07:36.185 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c109f70) proc(0x7f431c134ba0) pid(22843/22843) version 11:1:1:4 [ CSSD]2013-05-23 18:07:36.191 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(1/0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:36.191 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x7f431c0e6e30) [ CSSD]2013-05-23 18:07:36.191 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:07:36.191 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 18:07:36.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:36.467 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:36.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:36.968 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015691, gin 1, member 0 [ CSSD]2013-05-23 18:07:36.968 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:36.968 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:36.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:36.975 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:36.975 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:37.060 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:37.922 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c134ba0), client(2/0x7f431c13f030) [ CSSD]2013-05-23 18:07:37.922 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c13f030) [ CSSD]2013-05-23 18:07:37.922 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 18:07:37.922 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 23 [ CSSD]2013-05-23 18:07:38.513 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:38.513 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:39.112 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:40.506 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:07:40.507 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1176 (0x7f431bfe3c10) [ CSSD]2013-05-23 18:07:40.507 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3c10 (RPC#1176) tag(498002d) sent to node 1 [ CSSD]2013-05-23 18:07:40.507 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:40.507 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1176 (0x7f431bfe3c10) [ CSSD]2013-05-23 18:07:40.507 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3c10 (RPC#1176) state 6, flags 0x202 [ CSSD]2013-05-23 18:07:40.507 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3c10, ret 0, client (nil) [ CSSD]2013-05-23 18:07:40.508 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1176 [ CSSD]2013-05-23 18:07:40.562 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:40.562 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:40.606 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:40.606 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:41.135 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:41.331 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c12b920), client(2/0x7f431c0cf720) [ CSSD]2013-05-23 18:07:41.331 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x7f431c0cf720) [ CSSD]2013-05-23 18:07:41.331 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 18:07:41.331 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 24 [ CSSD]2013-05-23 18:07:42.109 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091910) allocated [ CSSD]2013-05-23 18:07:42.109 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:07:42.110 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091910 - 1,2,3 [ CSSD]2013-05-23 18:07:42.110 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c120e80) proc(0x7f431c091910) pid(23845) version 11:1:1:4 [ CSSD]2013-05-23 18:07:42.122 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c091910), client(1/0x7f431c092010) [ CSSD]2013-05-23 18:07:42.122 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c092010) [ CSSD]2013-05-23 18:07:42.122 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c092010 with con 0x7f431c091b10, requested num -1 [ CSSD]2013-05-23 18:07:42.122 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:42.122 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1177 (0x7f431bfe3c90) [ CSSD]2013-05-23 18:07:42.122 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3c90 (RPC#1177) tag(499002d) sent to node 1 [ CSSD]2013-05-23 18:07:42.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(0499002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:42.122 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015692, gin 1, member 0 [ CSSD]2013-05-23 18:07:42.122 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c092180/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:07:42.123 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:07:42.123 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:07:42.123 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3c90 (RPC#1177) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:42.123 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3c90, ret 0, client 0x7f431c092010 member 0x7f431c091800 [ CSSD]2013-05-23 18:07:42.123 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c092010 [ CSSD]2013-05-23 18:07:42.123 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1177 [ CSSD]2013-05-23 18:07:42.130 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c092010) [ CSSD]2013-05-23 18:07:42.130 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c092010), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:42.130 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c091800) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:42.130 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1178 (0x7f431bfe3d10) [ CSSD]2013-05-23 18:07:42.130 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3d10 (RPC#1178) tag(49a002d) sent to node 1 [ CSSD]2013-05-23 18:07:42.130 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(049a002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:42.130 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:42.130 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3d10 (RPC#1178) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:42.130 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3d10, ret 0, client 0x7f431c092010 member 0x7f431c091800 [ CSSD]2013-05-23 18:07:42.131 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1178 [ CSSD]2013-05-23 18:07:42.131 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c091800) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:07:42.333 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091910 [ CSSD]2013-05-23 18:07:42.333 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091910) con(0x7f431c120e80) skgpid ospid 23845 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:42.613 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:42.613 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:43.141 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:43.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:43.199 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015693, gin 1, member 0 [ CSSD]2013-05-23 18:07:43.199 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:43.199 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:43.206 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:43.206 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:43.206 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:44.619 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:44.619 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:45.155 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:45.192 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:45.192 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015694, gin 1, member 0 [ CSSD]2013-05-23 18:07:45.192 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:45.192 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:45.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:45.199 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:45.199 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:45.565 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:45.565 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:46.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:46.630 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:47.055 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c091840) allocated [ CSSD]2013-05-23 18:07:47.056 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:07:47.057 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c091840 - 1,2,3 [ CSSD]2013-05-23 18:07:47.057 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c1347e0) proc(0x7f431c091840) pid(23975) version 11:1:1:4 [ CSSD]2013-05-23 18:07:47.063 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x7f431c091840), client(1/0x7f431c091f40) [ CSSD]2013-05-23 18:07:47.063 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c091f40) [ CSSD]2013-05-23 18:07:47.063 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c091f40 with con 0x7f431c091a40, requested num -1 [ CSSD]2013-05-23 18:07:47.063 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:47.063 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1179 (0x7f431bfe3d90) [ CSSD]2013-05-23 18:07:47.063 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3d90 (RPC#1179) tag(49b002d) sent to node 1 [ CSSD]2013-05-23 18:07:47.063 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(049b002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015695, gin 1, member 0 [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c0920b0/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3d90 (RPC#1179) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3d90, ret 0, client 0x7f431c091f40 member 0x7f431c091730 [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c091f40 [ CSSD]2013-05-23 18:07:47.064 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1179 [ CSSD]2013-05-23 18:07:47.070 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c091f40) [ CSSD]2013-05-23 18:07:47.070 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c091f40), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:47.070 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c091730) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:47.070 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1180 (0x7f431bfe3e10) [ CSSD]2013-05-23 18:07:47.070 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3e10 (RPC#1180) tag(49c002d) sent to node 1 [ CSSD]2013-05-23 18:07:47.070 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(049c002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:47.070 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:47.070 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3e10 (RPC#1180) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:47.070 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3e10, ret 0, client 0x7f431c091f40 member 0x7f431c091730 [ CSSD]2013-05-23 18:07:47.070 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1180 [ CSSD]2013-05-23 18:07:47.070 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c091730) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:07:47.161 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:47.273 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c091840 [ CSSD]2013-05-23 18:07:47.273 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c091840) con(0x7f431c1347e0) skgpid ospid 23975 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:48.637 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:48.637 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:49.182 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:50.532 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:07:50.532 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1181 (0x7f431bfe3e90) [ CSSD]2013-05-23 18:07:50.533 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3e90 (RPC#1181) tag(49d002d) sent to node 1 [ CSSD]2013-05-23 18:07:50.533 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:50.533 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1181 (0x7f431bfe3e90) [ CSSD]2013-05-23 18:07:50.533 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3e90 (RPC#1181) state 6, flags 0x202 [ CSSD]2013-05-23 18:07:50.533 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe3e90, ret 0, client (nil) [ CSSD]2013-05-23 18:07:50.533 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1181 [ CSSD]2013-05-23 18:07:50.567 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:50.568 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:50.647 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:50.647 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:50.958 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c0e5910) [ CSSD]2013-05-23 18:07:50.958 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:07:50.958 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c0e5910), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:07:50.958 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 18:07:50.958 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 18:07:50.958 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c0840c0) skgpid 18620 ospid 18620 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:51.188 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:52.224 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:52.224 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015696, gin 1, member 0 [ CSSD]2013-05-23 18:07:52.224 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:07:52.224 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:07:52.230 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:52.230 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:07:52.230 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:07:52.653 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:52.653 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:53.194 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:54.660 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:54.660 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:55.236 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:55.570 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:55.571 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:07:56.709 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:56.709 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:57.242 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:57.365 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c120b90) allocated [ CSSD]2013-05-23 18:07:57.366 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:07:57.367 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c120b90 - 1,2,3 [ CSSD]2013-05-23 18:07:57.367 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c120b90) pid(24088) version 11:1:1:4 [ CSSD]2013-05-23 18:07:57.372 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c120b90), client(1/0x7f431c121240) [ CSSD]2013-05-23 18:07:57.372 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121240) [ CSSD]2013-05-23 18:07:57.373 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121240 with con 0x7f431c1347e0, requested num -1 [ CSSD]2013-05-23 18:07:57.373 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:57.373 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1182 (0x7f431bfe3f10) [ CSSD]2013-05-23 18:07:57.373 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3f10 (RPC#1182) tag(49e002d) sent to node 1 [ CSSD]2013-05-23 18:07:57.373 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(049e002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:57.373 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015697, gin 1, member 0 [ CSSD]2013-05-23 18:07:57.373 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133000/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:07:57.373 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:07:57.373 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:07:57.373 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3f10 (RPC#1182) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:57.374 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe3f10, ret 0, client 0x7f431c121240 member 0x7f431c0e5840 [ CSSD]2013-05-23 18:07:57.374 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121240 [ CSSD]2013-05-23 18:07:57.374 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1182 [ CSSD]2013-05-23 18:07:57.379 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121240) [ CSSD]2013-05-23 18:07:57.379 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121240), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:57.379 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:07:57.379 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1183 (0x7f431bfe3f90) [ CSSD]2013-05-23 18:07:57.380 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe3f90 (RPC#1183) tag(49f002d) sent to node 1 [ CSSD]2013-05-23 18:07:57.380 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(049f002d) incarnation(174631051) [ CSSD]2013-05-23 18:07:57.380 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:07:57.380 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe3f90 (RPC#1183) state 6, flags 0x100 [ CSSD]2013-05-23 18:07:57.380 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe3f90, ret 0, client 0x7f431c121240 member 0x7f431c0e5840 [ CSSD]2013-05-23 18:07:57.380 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1183 [ CSSD]2013-05-23 18:07:57.380 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:07:57.583 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c120b90 [ CSSD]2013-05-23 18:07:57.583 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c120b90) con(0x7f431c12a550) skgpid ospid 24088 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:07:58.734 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:07:58.734 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:07:59.253 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:07:59.573 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:07:59.573 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:08:00.556 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:08:00.557 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1184 (0x7f431bfe4010) [ CSSD]2013-05-23 18:08:00.557 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4010 (RPC#1184) tag(4a0002d) sent to node 1 [ CSSD]2013-05-23 18:08:00.557 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:00.557 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1184 (0x7f431bfe4010) [ CSSD]2013-05-23 18:08:00.557 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4010 (RPC#1184) state 6, flags 0x202 [ CSSD]2013-05-23 18:08:00.557 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4010, ret 0, client (nil) [ CSSD]2013-05-23 18:08:00.558 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1184 [ CSSD]2013-05-23 18:08:00.848 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:00.848 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:01.259 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:02.874 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:02.874 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:03.284 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:04.576 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:04.576 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:04.881 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:04.881 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:05.356 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:06.911 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:06.911 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:07.361 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:07.474 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:07.474 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015698, gin 1, member 0 [ CSSD]2013-05-23 18:08:07.474 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:08:07.474 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:08:07.481 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:07.481 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:08:07.481 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:08:08.917 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:08.917 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:09.373 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:09.579 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:09.579 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:10.582 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:08:10.583 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1185 (0x7f431bfe4090) [ CSSD]2013-05-23 18:08:10.583 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4090 (RPC#1185) tag(4a1002d) sent to node 1 [ CSSD]2013-05-23 18:08:10.583 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:10.583 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1185 (0x7f431bfe4090) [ CSSD]2013-05-23 18:08:10.583 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4090 (RPC#1185) state 6, flags 0x202 [ CSSD]2013-05-23 18:08:10.583 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4090, ret 0, client (nil) [ CSSD]2013-05-23 18:08:10.584 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1185 [ CSSD]2013-05-23 18:08:10.928 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:10.928 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:11.378 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:12.615 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 18:08:12.616 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:08:12.616 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 18:08:12.616 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c095ca0) proc(0x7f431c108da0) pid(24249) version 11:1:1:4 [ CSSD]2013-05-23 18:08:12.622 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108da0), client(1/0x7f431c135350) [ CSSD]2013-05-23 18:08:12.622 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135350) [ CSSD]2013-05-23 18:08:12.622 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135350 with con 0x7f431c0cf470, requested num -1 [ CSSD]2013-05-23 18:08:12.622 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:12.622 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1186 (0x7f431bfe4110) [ CSSD]2013-05-23 18:08:12.622 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4110 (RPC#1186) tag(4a2002d) sent to node 1 [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04a2002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015699, gin 1, member 0 [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133000/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4110 (RPC#1186) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4110, ret 0, client 0x7f431c135350 member 0x7f431c13c320 [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135350 [ CSSD]2013-05-23 18:08:12.623 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1186 [ CSSD]2013-05-23 18:08:12.630 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135350) [ CSSD]2013-05-23 18:08:12.630 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135350), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:12.630 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c320) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:12.630 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1187 (0x7f431bfe4190) [ CSSD]2013-05-23 18:08:12.630 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4190 (RPC#1187) tag(4a3002d) sent to node 1 [ CSSD]2013-05-23 18:08:12.630 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04a3002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:12.630 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:12.630 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4190 (RPC#1187) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:12.630 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4190, ret 0, client 0x7f431c135350 member 0x7f431c13c320 [ CSSD]2013-05-23 18:08:12.631 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1187 [ CSSD]2013-05-23 18:08:12.631 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c320) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:08:12.833 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 18:08:12.833 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c095ca0) skgpid ospid 24249 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:08:12.954 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:12.954 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:13.384 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:14.582 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:14.582 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:15.005 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:15.005 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:15.437 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:17.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:17.012 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:17.442 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:19.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:19.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:19.454 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:19.585 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:19.586 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:20.637 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:08:20.637 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1188 (0x7f431bfe4210) [ CSSD]2013-05-23 18:08:20.638 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4210 (RPC#1188) tag(4a4002d) sent to node 1 [ CSSD]2013-05-23 18:08:20.638 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:20.638 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1188 (0x7f431bfe4210) [ CSSD]2013-05-23 18:08:20.638 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4210 (RPC#1188) state 6, flags 0x202 [ CSSD]2013-05-23 18:08:20.638 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4210, ret 0, client (nil) [ CSSD]2013-05-23 18:08:20.638 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1188 [ CSSD]2013-05-23 18:08:21.028 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:21.028 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:21.460 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:22.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:22.725 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015700, gin 1, member 0 [ CSSD]2013-05-23 18:08:22.725 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:08:22.725 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:08:22.732 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:22.732 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:08:22.732 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:08:23.054 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:23.054 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:23.464 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:23.588 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:23.588 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:08:25.106 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:25.106 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:25.494 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:27.112 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:27.112 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:27.499 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:27.865 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c120b90) allocated [ CSSD]2013-05-23 18:08:27.866 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:08:27.867 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c120b90 - 1,2,3 [ CSSD]2013-05-23 18:08:27.867 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c120b90) pid(24392) version 11:1:1:4 [ CSSD]2013-05-23 18:08:27.872 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c120b90), client(1/0x7f431c135350) [ CSSD]2013-05-23 18:08:27.872 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c135350) [ CSSD]2013-05-23 18:08:27.872 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c135350 with con 0x7f431c0cf470, requested num -1 [ CSSD]2013-05-23 18:08:27.872 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:27.872 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1189 (0x7f431bfe4290) [ CSSD]2013-05-23 18:08:27.872 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4290 (RPC#1189) tag(4a5002d) sent to node 1 [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04a5002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015701, gin 1, member 0 [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133000/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4290 (RPC#1189) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4290, ret 0, client 0x7f431c135350 member 0x7f431c13c230 [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c135350 [ CSSD]2013-05-23 18:08:27.873 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1189 [ CSSD]2013-05-23 18:08:27.879 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c135350) [ CSSD]2013-05-23 18:08:27.879 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c135350), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:27.879 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c13c230) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:27.879 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1190 (0x7f431bfe4310) [ CSSD]2013-05-23 18:08:27.879 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4310 (RPC#1190) tag(4a6002d) sent to node 1 [ CSSD]2013-05-23 18:08:27.879 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04a6002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:27.879 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:27.879 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4310 (RPC#1190) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:27.879 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4310, ret 0, client 0x7f431c135350 member 0x7f431c13c230 [ CSSD]2013-05-23 18:08:27.879 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1190 [ CSSD]2013-05-23 18:08:27.879 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c13c230) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:08:28.082 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c120b90 [ CSSD]2013-05-23 18:08:28.082 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c120b90) con(0x7f431c12a550) skgpid ospid 24392 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:08:28.591 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:28.592 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:29.118 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:29.118 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:29.534 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:30.724 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:08:30.724 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1191 (0x7f431bfe4390) [ CSSD]2013-05-23 18:08:30.724 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4390 (RPC#1191) tag(4a7002d) sent to node 1 [ CSSD]2013-05-23 18:08:30.725 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:30.725 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1191 (0x7f431bfe4390) [ CSSD]2013-05-23 18:08:30.725 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4390 (RPC#1191) state 6, flags 0x202 [ CSSD]2013-05-23 18:08:30.725 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4390, ret 0, client (nil) [ CSSD]2013-05-23 18:08:30.725 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1191 [ CSSD]2013-05-23 18:08:31.129 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:31.129 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:31.539 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:33.135 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:33.135 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:33.551 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:33.595 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:33.595 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:35.157 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:35.157 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:35.556 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:37.164 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:37.164 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:37.580 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:37.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:37.980 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015702, gin 1, member 0 [ CSSD]2013-05-23 18:08:37.980 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:08:37.980 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:08:37.987 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:37.987 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:08:37.987 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:08:38.598 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:38.598 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:39.170 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:39.170 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:39.635 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:40.758 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:08:40.758 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1192 (0x7f431bfe4410) [ CSSD]2013-05-23 18:08:40.758 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4410 (RPC#1192) tag(4a8002d) sent to node 1 [ CSSD]2013-05-23 18:08:40.759 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:40.759 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1192 (0x7f431bfe4410) [ CSSD]2013-05-23 18:08:40.759 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4410 (RPC#1192) state 6, flags 0x202 [ CSSD]2013-05-23 18:08:40.759 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4410, ret 0, client (nil) [ CSSD]2013-05-23 18:08:40.759 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1192 [ CSSD]2013-05-23 18:08:41.220 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:41.220 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:41.639 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:42.601 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:42.601 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:08:43.114 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c108da0) allocated [ CSSD]2013-05-23 18:08:43.115 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:08:43.115 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c108da0 - 1,2,3 [ CSSD]2013-05-23 18:08:43.115 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c12a550) proc(0x7f431c108da0) pid(24529) version 11:1:1:4 [ CSSD]2013-05-23 18:08:43.121 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c108da0), client(1/0x7f431c121240) [ CSSD]2013-05-23 18:08:43.121 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121240) [ CSSD]2013-05-23 18:08:43.121 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121240 with con 0x7f431c1347e0, requested num -1 [ CSSD]2013-05-23 18:08:43.121 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:43.121 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1193 (0x7f431bfe4490) [ CSSD]2013-05-23 18:08:43.121 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4490 (RPC#1193) tag(4a9002d) sent to node 1 [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04a9002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015703, gin 1, member 0 [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133000/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4490 (RPC#1193) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4490, ret 0, client 0x7f431c121240 member 0x7f431c0e5840 [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121240 [ CSSD]2013-05-23 18:08:43.122 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1193 [ CSSD]2013-05-23 18:08:43.128 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121240) [ CSSD]2013-05-23 18:08:43.128 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121240), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:43.128 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:43.128 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1194 (0x7f431bfe4510) [ CSSD]2013-05-23 18:08:43.128 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4510 (RPC#1194) tag(4aa002d) sent to node 1 [ CSSD]2013-05-23 18:08:43.128 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04aa002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:43.128 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:43.128 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4510 (RPC#1194) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:43.128 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4510, ret 0, client 0x7f431c121240 member 0x7f431c0e5840 [ CSSD]2013-05-23 18:08:43.129 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1194 [ CSSD]2013-05-23 18:08:43.129 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:08:43.265 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:43.266 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:43.331 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108da0 [ CSSD]2013-05-23 18:08:43.331 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108da0) con(0x7f431c12a550) skgpid ospid 24529 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:08:43.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:43.463 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015704, gin 1, member 0 [ CSSD]2013-05-23 18:08:43.463 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:08:43.463 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:08:43.470 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:43.470 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:08:43.470 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:08:43.651 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:44.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:44.968 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, gid 2015705, gin 1, member 0 [ CSSD]2013-05-23 18:08:44.968 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1) [ CSSD]2013-05-23 18:08:44.968 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:08:44.975 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:44.975 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member 0 [ CSSD]2013-05-23 18:08:44.975 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac01.+asm1, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:08:45.103 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:08:45.104 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:08:45.105 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:08:45.105 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:08:45.106 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:08:45.107 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:08:45.267 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:45.267 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:45.657 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:47.273 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:47.273 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:47.313 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c120b90) allocated [ CSSD]2013-05-23 18:08:47.314 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:08:47.315 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c120b90 - 1,2,3 [ CSSD]2013-05-23 18:08:47.315 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c120b90) pid(24620) version 11:1:1:4 [ CSSD]2013-05-23 18:08:47.331 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x7f431c120b90), client(1/0x7f431c121240) [ CSSD]2013-05-23 18:08:47.331 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c121240) [ CSSD]2013-05-23 18:08:47.331 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c121240 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 18:08:47.331 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:47.331 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1195 (0x7f431bfe4590) [ CSSD]2013-05-23 18:08:47.331 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4590 (RPC#1195) tag(4ab002d) sent to node 1 [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ab002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015706, gin 1, member 0 [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c133000/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4590 (RPC#1195) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4590, ret 0, client 0x7f431c121240 member 0x7f431c0e5840 [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c121240 [ CSSD]2013-05-23 18:08:47.332 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1195 [ CSSD]2013-05-23 18:08:47.339 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c121240) [ CSSD]2013-05-23 18:08:47.339 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c121240), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:47.339 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c0e5840) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:47.339 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1196 (0x7f431bfe4610) [ CSSD]2013-05-23 18:08:47.339 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4610 (RPC#1196) tag(4ac002d) sent to node 1 [ CSSD]2013-05-23 18:08:47.339 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04ac002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:47.339 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:47.339 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4610 (RPC#1196) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:47.339 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4610, ret 0, client 0x7f431c121240 member 0x7f431c0e5840 [ CSSD]2013-05-23 18:08:47.339 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1196 [ CSSD]2013-05-23 18:08:47.339 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c0e5840) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:08:47.542 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c120b90 [ CSSD]2013-05-23 18:08:47.542 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c120b90) con(0x7f431c0840c0) skgpid ospid 24620 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:08:47.604 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:47.604 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:47.663 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:49.280 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:49.280 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:49.675 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:50.784 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:08:50.785 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1197 (0x7f431bfe4690) [ CSSD]2013-05-23 18:08:50.785 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4690 (RPC#1197) tag(4ad002d) sent to node 1 [ CSSD]2013-05-23 18:08:50.785 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:50.785 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1197 (0x7f431bfe4690) [ CSSD]2013-05-23 18:08:50.785 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4690 (RPC#1197) state 6, flags 0x202 [ CSSD]2013-05-23 18:08:50.785 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4690, ret 0, client (nil) [ CSSD]2013-05-23 18:08:50.785 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1197 [ CSSD]2013-05-23 18:08:51.331 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:51.331 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:51.681 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:52.607 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:52.607 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:53.228 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:53.228 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015707, gin 1, member 0 [ CSSD]2013-05-23 18:08:53.228 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:08:53.228 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:08:53.234 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:53.234 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:08:53.234 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:08:53.356 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:53.356 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:53.732 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:55.367 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:55.367 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:55.491 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:08:55.491 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:08:55.491 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c12e6d0), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:08:55.491 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 28 [ CSSD]2013-05-23 18:08:55.491 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7a00 [ CSSD]2013-05-23 18:08:55.491 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7a00) con(0x22dacd0) skgpid 19308 ospid 19308 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:08:55.738 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:57.393 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:57.393 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:57.610 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:08:57.611 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:08:57.744 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:08:58.365 [1191196992] >TRACE: clssgmAllocProc: (0x7f431c120b90) allocated [ CSSD]2013-05-23 18:08:58.366 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:08:58.366 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x7f431c120b90 - 1,2,3 [ CSSD]2013-05-23 18:08:58.366 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x7f431c0840c0) proc(0x7f431c120b90) pid(24724) version 11:1:1:4 [ CSSD]2013-05-23 18:08:58.382 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x7f431c120b90), client(1/0x7f431c12e6d0) [ CSSD]2013-05-23 18:08:58.382 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:08:58.382 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x7f431c12e6d0 with con 0x7f431c12a550, requested num -1 [ CSSD]2013-05-23 18:08:58.383 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:58.383 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1198 (0x7f431bfe4710) [ CSSD]2013-05-23 18:08:58.383 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4710 (RPC#1198) tag(4ae002d) sent to node 1 [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ae002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015708, gin 1, member 0 [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x7f431c121240/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4710 (RPC#1198) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4710, ret 0, client 0x7f431c12e6d0 member 0x7f431c135290 [ CSSD]2013-05-23 18:08:58.383 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x7f431c12e6d0 [ CSSD]2013-05-23 18:08:58.384 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1198 [ CSSD]2013-05-23 18:08:58.390 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x7f431c12e6d0) [ CSSD]2013-05-23 18:08:58.390 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x7f431c12e6d0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:58.390 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x7f431c135290) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:08:58.390 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1199 (0x7f431bfe4790) [ CSSD]2013-05-23 18:08:58.390 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4790 (RPC#1199) tag(4af002d) sent to node 1 [ CSSD]2013-05-23 18:08:58.391 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04af002d) incarnation(174631051) [ CSSD]2013-05-23 18:08:58.391 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:08:58.391 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4790 (RPC#1199) state 6, flags 0x100 [ CSSD]2013-05-23 18:08:58.391 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4790, ret 0, client 0x7f431c12e6d0 member 0x7f431c135290 [ CSSD]2013-05-23 18:08:58.391 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1199 [ CSSD]2013-05-23 18:08:58.391 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x7f431c135290) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:08:58.594 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c120b90 [ CSSD]2013-05-23 18:08:58.594 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c120b90) con(0x7f431c0840c0) skgpid ospid 24724 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:08:59.400 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:08:59.400 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:08:59.756 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:00.809 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:09:00.810 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1200 (0x7f431bfe4810) [ CSSD]2013-05-23 18:09:00.810 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4810 (RPC#1200) tag(4b0002d) sent to node 1 [ CSSD]2013-05-23 18:09:00.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:00.810 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1200 (0x7f431bfe4810) [ CSSD]2013-05-23 18:09:00.810 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4810 (RPC#1200) state 6, flags 0x202 [ CSSD]2013-05-23 18:09:00.810 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4810, ret 0, client (nil) [ CSSD]2013-05-23 18:09:00.810 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1200 [ CSSD]2013-05-23 18:09:01.431 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:01.431 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:01.613 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:01.613 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:09:01.762 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:03.457 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:03.457 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:03.782 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:05.467 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:05.467 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:05.788 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:06.615 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:06.616 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:07.472 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:07.472 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:07.793 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:08.499 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:08.499 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015709, gin 1, member 0 [ CSSD]2013-05-23 18:09:08.499 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:09:08.499 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:09:08.506 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:08.507 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:09:08.507 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:09:09.478 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:09.478 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:09.846 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:10.836 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:09:10.836 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1201 (0x7f431bfe4890) [ CSSD]2013-05-23 18:09:10.837 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4890 (RPC#1201) tag(4b1002d) sent to node 1 [ CSSD]2013-05-23 18:09:10.837 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:10.837 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1201 (0x7f431bfe4890) [ CSSD]2013-05-23 18:09:10.837 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4890 (RPC#1201) state 6, flags 0x202 [ CSSD]2013-05-23 18:09:10.837 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4890, ret 0, client (nil) [ CSSD]2013-05-23 18:09:10.837 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1201 [ CSSD]2013-05-23 18:09:11.532 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:11.532 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:11.619 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:11.619 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:11.871 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:13.539 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:13.539 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:13.626 [1191196992] >TRACE: clssgmAllocProc: (0x22fc9c0) allocated [ CSSD]2013-05-23 18:09:13.626 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:09:13.627 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fc9c0 - 1,2,3 [ CSSD]2013-05-23 18:09:13.627 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x230f170) proc(0x22fc9c0) pid(24863) version 11:1:1:4 [ CSSD]2013-05-23 18:09:13.632 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22fc9c0), client(1/0x2312950) [ CSSD]2013-05-23 18:09:13.632 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:09:13.632 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x22d7da0, requested num -1 [ CSSD]2013-05-23 18:09:13.632 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:13.632 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1202 (0x7f431bfe4910) [ CSSD]2013-05-23 18:09:13.632 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4910 (RPC#1202) tag(4b2002d) sent to node 1 [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04b2002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015710, gin 1, member 0 [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4910 (RPC#1202) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4910, ret 0, client 0x2312950 member 0x231d6c0 [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 18:09:13.633 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1202 [ CSSD]2013-05-23 18:09:13.638 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:09:13.638 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:13.638 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x231d6c0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:13.638 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1203 (0x7f431bfe4990) [ CSSD]2013-05-23 18:09:13.639 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4990 (RPC#1203) tag(4b3002d) sent to node 1 [ CSSD]2013-05-23 18:09:13.639 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04b3002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:13.639 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:13.639 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4990 (RPC#1203) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:13.639 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4990, ret 0, client 0x2312950 member 0x231d6c0 [ CSSD]2013-05-23 18:09:13.639 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1203 [ CSSD]2013-05-23 18:09:13.639 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x231d6c0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:09:13.842 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fc9c0 [ CSSD]2013-05-23 18:09:13.842 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fc9c0) con(0x230f170) skgpid ospid 24863 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:09:13.883 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:15.549 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:15.549 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:15.889 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:16.621 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:16.622 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:17.594 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:17.594 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:17.895 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:19.600 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:19.600 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:19.947 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:20.861 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:09:20.862 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1204 (0x7f431bfe4a10) [ CSSD]2013-05-23 18:09:20.862 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4a10 (RPC#1204) tag(4b4002d) sent to node 1 [ CSSD]2013-05-23 18:09:20.862 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:20.862 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1204 (0x7f431bfe4a10) [ CSSD]2013-05-23 18:09:20.862 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4a10 (RPC#1204) state 6, flags 0x202 [ CSSD]2013-05-23 18:09:20.862 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4a10, ret 0, client (nil) [ CSSD]2013-05-23 18:09:20.862 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1204 [ CSSD]2013-05-23 18:09:21.624 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:21.625 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:21.632 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:21.632 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:21.953 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:23.639 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:23.639 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:23.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:23.762 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015711, gin 1, member 0 [ CSSD]2013-05-23 18:09:23.762 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:09:23.762 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:09:23.769 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:23.769 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:09:23.769 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x2302f30) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:09:23.964 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:25.627 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:25.627 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:09:25.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:25.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:25.970 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:27.655 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:27.655 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:27.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:28.884 [1191196992] >TRACE: clssgmAllocProc: (0x22fdb60) allocated [ CSSD]2013-05-23 18:09:28.885 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:09:28.885 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22fdb60 - 1,2,3 [ CSSD]2013-05-23 18:09:28.885 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22fdb60) pid(25008) version 11:1:1:4 [ CSSD]2013-05-23 18:09:28.891 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22fdb60), client(1/0x22e01e0) [ CSSD]2013-05-23 18:09:28.891 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:09:28.891 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e01e0 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 18:09:28.891 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:28.891 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1205 (0x7f431bfe4a90) [ CSSD]2013-05-23 18:09:28.891 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4a90 (RPC#1205) tag(4b5002d) sent to node 1 [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04b5002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015712, gin 1, member 0 [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4a90 (RPC#1205) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4a90, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e01e0 [ CSSD]2013-05-23 18:09:28.892 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1205 [ CSSD]2013-05-23 18:09:28.898 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:09:28.898 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e01e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:28.898 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:28.898 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1206 (0x7f431bfe4b10) [ CSSD]2013-05-23 18:09:28.898 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4b10 (RPC#1206) tag(4b6002d) sent to node 1 [ CSSD]2013-05-23 18:09:28.898 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04b6002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:28.898 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:28.898 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4b10 (RPC#1206) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:28.898 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4b10, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:09:28.899 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1206 [ CSSD]2013-05-23 18:09:28.899 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:09:29.101 [1191196992] >TRACE: clssgmDeadProc: proc 0x22fdb60 [ CSSD]2013-05-23 18:09:29.101 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22fdb60) con(0x2312880) skgpid ospid 25008 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:09:29.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:29.661 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:29.987 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:30.629 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:30.629 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:30.887 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:09:30.887 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1207 (0x7f431bfe4b90) [ CSSD]2013-05-23 18:09:30.887 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4b90 (RPC#1207) tag(4b7002d) sent to node 1 [ CSSD]2013-05-23 18:09:30.888 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:30.888 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1207 (0x7f431bfe4b90) [ CSSD]2013-05-23 18:09:30.888 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4b90 (RPC#1207) state 6, flags 0x202 [ CSSD]2013-05-23 18:09:30.888 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4b90, ret 0, client (nil) [ CSSD]2013-05-23 18:09:30.888 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1207 [ CSSD]2013-05-23 18:09:31.671 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:31.671 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:32.012 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:32.423 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:09:32.423 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:09:32.424 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:09:32.425 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:09:32.503 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:32.503 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock ORA_CLSRCSN_SRV_HWPROD1, gid 2015713, gin 1, member 0 [ CSSD]2013-05-23 18:09:32.503 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1002) node(1) grock (0x7f431c083bb0/ORA_CLSRCSN_SRV_HWPROD1) [ CSSD]2013-05-23 18:09:32.503 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock ORA_CLSRCSN_SRV_HWPROD1 member(0/Remote) node(1) flags 0x1002 0x1000002 [ CSSD]2013-05-23 18:09:32.511 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:32.511 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock ORA_CLSRCSN_SRV_HWPROD1, member 0 [ CSSD]2013-05-23 18:09:32.511 [1104931136] >TRACE: clssgmRemoveMember: grock ORA_CLSRCSN_SRV_HWPROD1, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:09:33.677 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:33.677 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:34.044 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:35.632 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:35.632 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:35.730 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:35.730 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:36.050 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:37.736 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:37.736 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:38.056 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:39.022 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:39.022 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015714, gin 1, member 0 [ CSSD]2013-05-23 18:09:39.022 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:09:39.023 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:09:39.030 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:39.030 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:09:39.030 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c120b90) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:09:39.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:39.762 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:40.067 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:40.635 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:40.636 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:40.914 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:09:40.914 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1208 (0x7f431bfe4c10) [ CSSD]2013-05-23 18:09:40.914 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4c10 (RPC#1208) tag(4b8002d) sent to node 1 [ CSSD]2013-05-23 18:09:40.915 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:40.915 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1208 (0x7f431bfe4c10) [ CSSD]2013-05-23 18:09:40.915 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4c10 (RPC#1208) state 6, flags 0x202 [ CSSD]2013-05-23 18:09:40.915 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4c10, ret 0, client (nil) [ CSSD]2013-05-23 18:09:40.915 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1208 [ CSSD]2013-05-23 18:09:41.772 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:41.772 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:42.092 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:43.715 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:43.715 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015715, gin 1, member 0 [ CSSD]2013-05-23 18:09:43.715 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:09:43.715 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:09:43.721 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:43.721 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:09:43.721 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:09:43.779 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:43.779 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:44.133 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 18:09:44.134 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:09:44.134 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 18:09:44.134 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22dacd0) pid(25151) version 11:1:1:4 [ CSSD]2013-05-23 18:09:44.145 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:44.151 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x2312950) [ CSSD]2013-05-23 18:09:44.151 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:09:44.151 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x2312950 with con 0x231d790, requested num -1 [ CSSD]2013-05-23 18:09:44.151 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:44.151 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1209 (0x7f431bfe4c90) [ CSSD]2013-05-23 18:09:44.151 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4c90 (RPC#1209) tag(4b9002d) sent to node 1 [ CSSD]2013-05-23 18:09:44.151 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04b9002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015716, gin 1, member 0 [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4c90 (RPC#1209) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4c90, ret 0, client 0x2312950 member 0x2303710 [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x2312950 [ CSSD]2013-05-23 18:09:44.152 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1209 [ CSSD]2013-05-23 18:09:44.159 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:09:44.159 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x2312950), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:44.159 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:44.159 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1210 (0x7f431bfe4d10) [ CSSD]2013-05-23 18:09:44.159 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4d10 (RPC#1210) tag(4ba002d) sent to node 1 [ CSSD]2013-05-23 18:09:44.159 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04ba002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:44.159 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:44.159 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4d10 (RPC#1210) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:44.159 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4d10, ret 0, client 0x2312950 member 0x2303710 [ CSSD]2013-05-23 18:09:44.159 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1210 [ CSSD]2013-05-23 18:09:44.160 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:09:44.362 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 18:09:44.362 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x205ecb0) skgpid ospid 25151 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:09:45.638 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:45.639 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:45.831 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:45.831 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:46.150 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:47.584 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 18:09:47.585 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:09:47.585 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 18:09:47.585 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22d7cd0) pid(25215) version 11:1:1:4 [ CSSD]2013-05-23 18:09:47.591 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7cd0), client(1/0x22e01e0) [ CSSD]2013-05-23 18:09:47.591 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:09:47.591 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e01e0 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 18:09:47.591 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:47.591 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1211 (0x7f431bfe4d90) [ CSSD]2013-05-23 18:09:47.592 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4d90 (RPC#1211) tag(4bb002d) sent to node 1 [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04bb002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015717, gin 1, member 0 [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4d90 (RPC#1211) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4d90, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e01e0 [ CSSD]2013-05-23 18:09:47.592 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1211 [ CSSD]2013-05-23 18:09:47.598 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:09:47.598 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e01e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:47.598 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:47.598 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1212 (0x7f431bfe4e10) [ CSSD]2013-05-23 18:09:47.598 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4e10 (RPC#1212) tag(4bc002d) sent to node 1 [ CSSD]2013-05-23 18:09:47.598 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04bc002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:47.598 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:47.599 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4e10 (RPC#1212) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:47.599 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4e10, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:09:47.599 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1212 [ CSSD]2013-05-23 18:09:47.599 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:09:47.801 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 18:09:47.801 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2312880) skgpid ospid 25215 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:09:47.836 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:47.837 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:48.156 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:49.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:49.843 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:50.168 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:50.642 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:50.642 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:50.940 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:09:50.941 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1213 (0x7f431bfe4e90) [ CSSD]2013-05-23 18:09:50.941 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4e90 (RPC#1213) tag(4bd002d) sent to node 1 [ CSSD]2013-05-23 18:09:50.942 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:50.942 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1213 (0x7f431bfe4e90) [ CSSD]2013-05-23 18:09:50.942 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4e90 (RPC#1213) state 6, flags 0x202 [ CSSD]2013-05-23 18:09:50.942 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe4e90, ret 0, client (nil) [ CSSD]2013-05-23 18:09:50.942 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1213 [ CSSD]2013-05-23 18:09:51.853 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:51.853 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:52.193 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:53.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:53.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:54.245 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:54.301 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:54.301 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015718, gin 1, member 0 [ CSSD]2013-05-23 18:09:54.301 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:09:54.301 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:09:54.308 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:54.308 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:09:54.308 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c120b90) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:09:55.644 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:09:55.644 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:09:55.871 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:55.871 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:56.251 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:57.877 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:57.877 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:09:58.257 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:09:59.393 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 18:09:59.394 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:09:59.394 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 18:09:59.394 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22dacd0) pid(25305) version 11:1:1:4 [ CSSD]2013-05-23 18:09:59.410 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x22e01e0) [ CSSD]2013-05-23 18:09:59.410 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:09:59.410 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e01e0 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 18:09:59.410 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:59.410 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1214 (0x7f431bfe4f10) [ CSSD]2013-05-23 18:09:59.410 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4f10 (RPC#1214) tag(4be002d) sent to node 1 [ CSSD]2013-05-23 18:09:59.410 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04be002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:59.410 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015719, gin 1, member 0 [ CSSD]2013-05-23 18:09:59.410 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:09:59.411 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:09:59.411 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:09:59.411 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4f10 (RPC#1214) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:59.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe4f10, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:09:59.411 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e01e0 [ CSSD]2013-05-23 18:09:59.411 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1214 [ CSSD]2013-05-23 18:09:59.417 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:09:59.417 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e01e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:59.417 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:09:59.417 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1215 (0x7f431bfe4f90) [ CSSD]2013-05-23 18:09:59.417 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe4f90 (RPC#1215) tag(4bf002d) sent to node 1 [ CSSD]2013-05-23 18:09:59.418 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04bf002d) incarnation(174631051) [ CSSD]2013-05-23 18:09:59.418 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:09:59.418 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe4f90 (RPC#1215) state 6, flags 0x100 [ CSSD]2013-05-23 18:09:59.418 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe4f90, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:09:59.418 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1215 [ CSSD]2013-05-23 18:09:59.418 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:09:59.621 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 18:09:59.621 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2312880) skgpid ospid 25305 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:09:59.883 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:09:59.883 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:00.268 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:00.647 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:00.647 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:00.967 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:10:00.967 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1216 (0x7f431bfe5010) [ CSSD]2013-05-23 18:10:00.967 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5010 (RPC#1216) tag(4c0002d) sent to node 1 [ CSSD]2013-05-23 18:10:00.968 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:00.968 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1216 (0x7f431bfe5010) [ CSSD]2013-05-23 18:10:00.968 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5010 (RPC#1216) state 6, flags 0x202 [ CSSD]2013-05-23 18:10:00.968 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5010, ret 0, client (nil) [ CSSD]2013-05-23 18:10:00.968 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1216 [ CSSD]2013-05-23 18:10:01.934 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:01.934 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:02.292 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:03.980 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:03.980 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:04.346 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:05.650 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:05.651 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:06.012 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:06.012 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:06.352 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:07.997 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 18:10:07.998 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:10:07.999 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 18:10:07.999 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22d7cd0) pid(25597) version 11:1:1:4 [ CSSD]2013-05-23 18:10:08.018 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:08.018 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:08.358 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:08.593 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d7cd0 [ CSSD]2013-05-23 18:10:08.593 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d7cd0) con(0x2312880) skgpid ospid 25597 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:10:09.553 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:09.553 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015720, gin 1, member 0 [ CSSD]2013-05-23 18:10:09.553 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:10:09.553 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:10:09.559 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:09.559 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:10:09.559 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:10:09.653 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:09.653 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:10:10.039 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:10.039 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:10.369 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:10.991 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:10:10.991 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1217 (0x7f431bfe5090) [ CSSD]2013-05-23 18:10:10.991 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5090 (RPC#1217) tag(4c1002d) sent to node 1 [ CSSD]2013-05-23 18:10:10.992 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:10.992 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1217 (0x7f431bfe5090) [ CSSD]2013-05-23 18:10:10.992 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5090 (RPC#1217) state 6, flags 0x202 [ CSSD]2013-05-23 18:10:10.992 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5090, ret 0, client (nil) [ CSSD]2013-05-23 18:10:10.992 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1217 [ CSSD]2013-05-23 18:10:12.045 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:12.045 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:12.414 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:14.051 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:14.051 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:14.465 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:14.652 [1191196992] >TRACE: clssgmAllocProc: (0x22dacd0) allocated [ CSSD]2013-05-23 18:10:14.654 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:10:14.654 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22dacd0 - 1,2,3 [ CSSD]2013-05-23 18:10:14.654 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x2312880) proc(0x22dacd0) pid(25745) version 11:1:1:4 [ CSSD]2013-05-23 18:10:14.657 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:14.657 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:14.660 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22dacd0), client(1/0x22e01e0) [ CSSD]2013-05-23 18:10:14.660 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:10:14.660 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22e01e0 with con 0x22e2f80, requested num -1 [ CSSD]2013-05-23 18:10:14.660 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:14.660 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1218 (0x7f431bfe5110) [ CSSD]2013-05-23 18:10:14.661 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5110 (RPC#1218) tag(4c2002d) sent to node 1 [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04c2002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015721, gin 1, member 0 [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22dfd50/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5110 (RPC#1218) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5110, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22e01e0 [ CSSD]2013-05-23 18:10:14.661 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1218 [ CSSD]2013-05-23 18:10:14.667 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22e01e0) [ CSSD]2013-05-23 18:10:14.667 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22e01e0), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:14.667 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x2303710) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:14.667 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1219 (0x7f431bfe5190) [ CSSD]2013-05-23 18:10:14.667 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5190 (RPC#1219) tag(4c3002d) sent to node 1 [ CSSD]2013-05-23 18:10:14.668 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04c3002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:14.668 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:14.668 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5190 (RPC#1219) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:14.668 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5190, ret 0, client 0x22e01e0 member 0x2303710 [ CSSD]2013-05-23 18:10:14.668 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1219 [ CSSD]2013-05-23 18:10:14.668 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x2303710) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:10:14.870 [1191196992] >TRACE: clssgmDeadProc: proc 0x22dacd0 [ CSSD]2013-05-23 18:10:14.870 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22dacd0) con(0x2312880) skgpid ospid 25745 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:10:16.062 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:16.062 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:16.470 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:18.068 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:18.068 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:18.474 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:19.660 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:19.660 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:20.099 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:20.099 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:20.488 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:21.015 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:10:21.015 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1220 (0x7f431bfe5210) [ CSSD]2013-05-23 18:10:21.015 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5210 (RPC#1220) tag(4c4002d) sent to node 1 [ CSSD]2013-05-23 18:10:21.016 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:21.016 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1220 (0x7f431bfe5210) [ CSSD]2013-05-23 18:10:21.016 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5210 (RPC#1220) state 6, flags 0x202 [ CSSD]2013-05-23 18:10:21.016 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5210, ret 0, client (nil) [ CSSD]2013-05-23 18:10:21.016 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1220 [ CSSD]2013-05-23 18:10:21.495 [1191196992] >TRACE: clssgmAllocProc: (0x22d7cd0) allocated [ CSSD]2013-05-23 18:10:21.496 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:10:21.497 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d7cd0 - 1,2,3 [ CSSD]2013-05-23 18:10:21.497 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205ecb0) proc(0x22d7cd0) pid(25836/25836) version 11:1:1:4 [ CSSD]2013-05-23 18:10:21.512 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7cd0), client(1/0x2312950) [ CSSD]2013-05-23 18:10:21.512 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x2312950) [ CSSD]2013-05-23 18:10:21.512 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:10:21.512 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 29 [ CSSD]2013-05-23 18:10:22.105 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:22.105 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:22.493 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:22.630 [1191196992] >TRACE: clssgmAllocProc: (0x22f8740) allocated [ CSSD]2013-05-23 18:10:22.631 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:10:22.632 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f8740 - 1,2,3 [ CSSD]2013-05-23 18:10:22.632 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22d7750) proc(0x22f8740) pid(24703/24703) version 11:1:1:4 [ CSSD]2013-05-23 18:10:22.648 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f8740), client(1/0x205ed60) [ CSSD]2013-05-23 18:10:22.648 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 1 (0x205ed60) [ CSSD]2013-05-23 18:10:22.648 [1191196992] >TRACE: clssgmRegisterShared: grp DG_LOCAL_DATA, mbr 0, type 1 [ CSSD]2013-05-23 18:10:22.648 [1191196992] >TRACE: clssgmRegisterShared: local grock DG_LOCAL_DATA member 0 share type 1, refcount 30 [ CSSD]2013-05-23 18:10:22.650 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22f8740), client(2/0x2302d30) [ CSSD]2013-05-23 18:10:22.650 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x2302d30) [ CSSD]2013-05-23 18:10:22.650 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 18:10:22.650 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 25 [ CSSD]2013-05-23 18:10:24.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:24.171 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:24.516 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:24.662 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:24.662 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:24.803 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:24.803 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015722, gin 1, member 0 [ CSSD]2013-05-23 18:10:24.803 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:10:24.803 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:10:24.810 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:24.810 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:10:24.810 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c120b90) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:10:26.243 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:26.243 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:26.521 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:26.572 [1191196992] >TRACE: clssgmRegisterClient: proc(57/0x22d7cd0), client(2/0x22d7350) [ CSSD]2013-05-23 18:10:26.572 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHREG recvd from client 2 (0x22d7350) [ CSSD]2013-05-23 18:10:26.572 [1191196992] >TRACE: clssgmRegisterShared: grp DBHWPROD, mbr 1, type 1 [ CSSD]2013-05-23 18:10:26.572 [1191196992] >TRACE: clssgmRegisterShared: global grock DBHWPROD member 1 share type 1, refcount 26 [ CSSD]2013-05-23 18:10:28.268 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:28.268 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:28.554 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:29.665 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:29.666 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:29.913 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ef0) allocated [ CSSD]2013-05-23 18:10:29.914 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:10:29.914 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ef0 - 1,2,3 [ CSSD]2013-05-23 18:10:29.914 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e3d0) proc(0x22d6ef0) pid(25906) version 11:1:1:4 [ CSSD]2013-05-23 18:10:29.919 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6ef0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:10:29.919 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:10:29.919 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:10:29.919 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:29.919 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1221 (0x7f431bfe5290) [ CSSD]2013-05-23 18:10:29.919 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5290 (RPC#1221) tag(4c5002d) sent to node 1 [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04c5002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015723, gin 1, member 0 [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5290 (RPC#1221) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5290, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:10:29.920 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1221 [ CSSD]2013-05-23 18:10:29.926 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:10:29.926 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:29.926 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d6cf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:29.926 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1222 (0x7f431bfe5310) [ CSSD]2013-05-23 18:10:29.926 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5310 (RPC#1222) tag(4c6002d) sent to node 1 [ CSSD]2013-05-23 18:10:29.926 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04c6002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:29.926 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:29.926 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5310 (RPC#1222) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:29.926 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5310, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:10:29.927 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1222 [ CSSD]2013-05-23 18:10:29.927 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d6cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:10:30.129 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ef0 [ CSSD]2013-05-23 18:10:30.129 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ef0) con(0x205e3d0) skgpid ospid 25906 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:10:30.340 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:30.340 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:30.559 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:31.041 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:10:31.042 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1223 (0x7f431bfe5390) [ CSSD]2013-05-23 18:10:31.042 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5390 (RPC#1223) tag(4c7002d) sent to node 1 [ CSSD]2013-05-23 18:10:31.042 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:31.042 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1223 (0x7f431bfe5390) [ CSSD]2013-05-23 18:10:31.042 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5390 (RPC#1223) state 6, flags 0x202 [ CSSD]2013-05-23 18:10:31.042 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5390, ret 0, client (nil) [ CSSD]2013-05-23 18:10:31.042 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1223 [ CSSD]2013-05-23 18:10:32.365 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:32.365 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:32.604 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:33.668 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:33.668 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:10:34.372 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:34.372 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:34.678 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:36.384 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:36.384 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:36.684 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:38.390 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:38.390 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:38.672 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:38.672 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:38.694 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:40.053 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:40.053 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015724, gin 1, member 0 [ CSSD]2013-05-23 18:10:40.053 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:10:40.053 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:10:40.059 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:40.059 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:10:40.059 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:10:40.441 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:40.441 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:40.699 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:41.071 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:10:41.071 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1224 (0x7f431bfe5410) [ CSSD]2013-05-23 18:10:41.071 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5410 (RPC#1224) tag(4c8002d) sent to node 1 [ CSSD]2013-05-23 18:10:41.072 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:41.072 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1224 (0x7f431bfe5410) [ CSSD]2013-05-23 18:10:41.072 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5410 (RPC#1224) state 6, flags 0x202 [ CSSD]2013-05-23 18:10:41.072 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5410, ret 0, client (nil) [ CSSD]2013-05-23 18:10:41.072 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1224 [ CSSD]2013-05-23 18:10:42.447 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:42.447 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:42.723 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:43.675 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:43.676 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:43.991 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:43.991 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015725, gin 1, member 0 [ CSSD]2013-05-23 18:10:43.991 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:10:43.991 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:10:43.998 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:43.998 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:10:43.998 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c120b90) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:10:44.454 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:44.454 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:44.758 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:45.161 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ef0) allocated [ CSSD]2013-05-23 18:10:45.162 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:10:45.162 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ef0 - 1,2,3 [ CSSD]2013-05-23 18:10:45.162 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e3d0) proc(0x22d6ef0) pid(26075) version 11:1:1:4 [ CSSD]2013-05-23 18:10:45.167 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6ef0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:10:45.167 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:10:45.167 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:10:45.167 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:45.167 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1225 (0x7f431bfe5490) [ CSSD]2013-05-23 18:10:45.167 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5490 (RPC#1225) tag(4c9002d) sent to node 1 [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04c9002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015726, gin 1, member 0 [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5490 (RPC#1225) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5490, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:10:45.168 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1225 [ CSSD]2013-05-23 18:10:45.174 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:10:45.174 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:45.174 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d6cf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:45.174 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1226 (0x7f431bfe5510) [ CSSD]2013-05-23 18:10:45.174 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5510 (RPC#1226) tag(4ca002d) sent to node 1 [ CSSD]2013-05-23 18:10:45.175 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04ca002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:45.175 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:45.175 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5510 (RPC#1226) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:45.175 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5510, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:10:45.175 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1226 [ CSSD]2013-05-23 18:10:45.175 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d6cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:10:45.377 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ef0 [ CSSD]2013-05-23 18:10:45.377 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ef0) con(0x205e3d0) skgpid ospid 26075 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:10:46.463 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:46.463 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:46.764 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:47.840 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ef0) allocated [ CSSD]2013-05-23 18:10:47.841 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:10:47.842 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ef0 - 1,2,3 [ CSSD]2013-05-23 18:10:47.842 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e3d0) proc(0x22d6ef0) pid(26143) version 11:1:1:4 [ CSSD]2013-05-23 18:10:47.848 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6ef0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:10:47.848 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:10:47.848 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:10:47.848 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:47.848 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1227 (0x7f431bfe5590) [ CSSD]2013-05-23 18:10:47.848 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5590 (RPC#1227) tag(4cb002d) sent to node 1 [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04cb002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015727, gin 1, member 0 [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5590 (RPC#1227) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5590, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:10:47.849 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1227 [ CSSD]2013-05-23 18:10:47.854 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:10:47.854 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:47.854 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d6cf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:10:47.854 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1228 (0x7f431bfe5610) [ CSSD]2013-05-23 18:10:47.854 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5610 (RPC#1228) tag(4cc002d) sent to node 1 [ CSSD]2013-05-23 18:10:47.855 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04cc002d) incarnation(174631051) [ CSSD]2013-05-23 18:10:47.855 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:10:47.855 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5610 (RPC#1228) state 6, flags 0x100 [ CSSD]2013-05-23 18:10:47.855 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5610, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:10:47.855 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1228 [ CSSD]2013-05-23 18:10:47.855 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d6cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:10:48.057 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ef0 [ CSSD]2013-05-23 18:10:48.057 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ef0) con(0x205e3d0) skgpid ospid 26143 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:10:48.469 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:48.469 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:48.679 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:48.679 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:48.776 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:50.541 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:50.541 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:50.800 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:51.093 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:10:51.094 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1229 (0x7f431bfe5690) [ CSSD]2013-05-23 18:10:51.094 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5690 (RPC#1229) tag(4cd002d) sent to node 1 [ CSSD]2013-05-23 18:10:51.094 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:51.095 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1229 (0x7f431bfe5690) [ CSSD]2013-05-23 18:10:51.095 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5690 (RPC#1229) state 6, flags 0x202 [ CSSD]2013-05-23 18:10:51.095 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5690, ret 0, client (nil) [ CSSD]2013-05-23 18:10:51.095 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1229 [ CSSD]2013-05-23 18:10:52.568 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:52.568 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:52.805 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:53.682 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:53.682 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:54.574 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:54.574 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:54.859 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:55.317 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:55.317 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015728, gin 1, member 0 [ CSSD]2013-05-23 18:10:55.317 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:10:55.317 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:10:55.324 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:55.324 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:10:55.324 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c108da0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:10:56.585 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:56.585 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:56.865 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:10:58.591 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:10:58.591 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:10:58.685 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:10:58.685 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:10:58.876 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:00.411 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ef0) allocated [ CSSD]2013-05-23 18:11:00.412 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:11:00.412 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ef0 - 1,2,3 [ CSSD]2013-05-23 18:11:00.412 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e3d0) proc(0x22d6ef0) pid(26240) version 11:1:1:4 [ CSSD]2013-05-23 18:11:00.419 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6ef0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:11:00.419 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:00.419 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:11:00.419 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:00.419 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1230 (0x7f431bfe5710) [ CSSD]2013-05-23 18:11:00.419 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5710 (RPC#1230) tag(4ce002d) sent to node 1 [ CSSD]2013-05-23 18:11:00.419 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04ce002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:00.419 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015729, gin 1, member 0 [ CSSD]2013-05-23 18:11:00.419 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:11:00.420 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:11:00.420 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:11:00.420 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5710 (RPC#1230) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:00.420 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5710, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:11:00.420 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:11:00.420 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1230 [ CSSD]2013-05-23 18:11:00.428 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:00.428 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:00.428 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d6cf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:00.428 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1231 (0x7f431bfe5790) [ CSSD]2013-05-23 18:11:00.428 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5790 (RPC#1231) tag(4cf002d) sent to node 1 [ CSSD]2013-05-23 18:11:00.429 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04cf002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:00.429 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:00.429 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5790 (RPC#1231) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:00.429 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5790, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:11:00.429 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1231 [ CSSD]2013-05-23 18:11:00.429 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d6cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:11:00.450 [1191196992] >TRACE: clssgmAllocProc: (0x22f3010) allocated [ CSSD]2013-05-23 18:11:00.451 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:11:00.451 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3010 - 1,2,3 [ CSSD]2013-05-23 18:11:00.451 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x22f3010) pid(26246) version 11:1:1:4 [ CSSD]2013-05-23 18:11:00.458 [1191196992] >TRACE: clssgmRegisterClient: proc(60/0x22f3010), client(1/0x22f1d30) [ CSSD]2013-05-23 18:11:00.458 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:00.458 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:11:00.458 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:00.458 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1232 (0x7f431bfe5810) [ CSSD]2013-05-23 18:11:00.458 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5810 (RPC#1232) tag(4d0002d) sent to node 1 [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d0002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015730, gin 1, member 0 [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22ec570/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5810 (RPC#1232) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5810, ret 0, client 0x22f1d30 member 0x22f4c00 [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:11:00.459 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1232 [ CSSD]2013-05-23 18:11:00.464 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:00.464 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:00.464 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f4c00) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:00.464 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1233 (0x7f431bfe5890) [ CSSD]2013-05-23 18:11:00.465 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5890 (RPC#1233) tag(4d1002d) sent to node 1 [ CSSD]2013-05-23 18:11:00.465 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04d1002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:00.465 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:00.465 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5890 (RPC#1233) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:00.465 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5890, ret 0, client 0x22f1d30 member 0x22f4c00 [ CSSD]2013-05-23 18:11:00.465 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1233 [ CSSD]2013-05-23 18:11:00.465 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f4c00) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:11:00.631 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ef0 [ CSSD]2013-05-23 18:11:00.631 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ef0) con(0x205e3d0) skgpid ospid 26240 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:00.642 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:00.642 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:00.667 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3010 [ CSSD]2013-05-23 18:11:00.667 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3010) con(0x22dffa0) skgpid ospid 26246 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:00.881 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:01.119 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:11:01.119 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1234 (0x7f431bfe5910) [ CSSD]2013-05-23 18:11:01.119 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5910 (RPC#1234) tag(4d2002d) sent to node 1 [ CSSD]2013-05-23 18:11:01.120 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:01.120 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1234 (0x7f431bfe5910) [ CSSD]2013-05-23 18:11:01.120 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5910 (RPC#1234) state 6, flags 0x202 [ CSSD]2013-05-23 18:11:01.120 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5910, ret 0, client (nil) [ CSSD]2013-05-23 18:11:01.120 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1234 [ CSSD]2013-05-23 18:11:02.648 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:02.648 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:02.886 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:03.688 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:03.689 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:11:04.654 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:04.654 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:04.959 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:06.666 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:06.666 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:06.965 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:08.672 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:08.672 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:08.692 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:08.692 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:11:08.976 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:10.579 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:10.579 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015731, gin 1, member 0 [ CSSD]2013-05-23 18:11:10.579 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:11:10.579 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:11:10.586 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:10.586 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:11:10.586 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c120b90) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:11:10.681 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:10.681 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:11.002 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:11.143 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:11:11.143 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1235 (0x7f431bfe5990) [ CSSD]2013-05-23 18:11:11.143 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5990 (RPC#1235) tag(4d3002d) sent to node 1 [ CSSD]2013-05-23 18:11:11.144 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:11.144 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1235 (0x7f431bfe5990) [ CSSD]2013-05-23 18:11:11.144 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5990 (RPC#1235) state 6, flags 0x202 [ CSSD]2013-05-23 18:11:11.144 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5990, ret 0, client (nil) [ CSSD]2013-05-23 18:11:11.144 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1235 [ CSSD]2013-05-23 18:11:12.687 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:12.687 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:13.008 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:13.695 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:13.696 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:11:14.694 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:14.694 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:15.060 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:15.668 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ef0) allocated [ CSSD]2013-05-23 18:11:15.669 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:11:15.670 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ef0 - 1,2,3 [ CSSD]2013-05-23 18:11:15.670 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e3d0) proc(0x22d6ef0) pid(26420) version 11:1:1:4 [ CSSD]2013-05-23 18:11:15.675 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6ef0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:11:15.675 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:15.675 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:11:15.675 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:15.675 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1236 (0x7f431bfe5a10) [ CSSD]2013-05-23 18:11:15.675 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5a10 (RPC#1236) tag(4d4002d) sent to node 1 [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d4002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015732, gin 1, member 0 [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3b30/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5a10 (RPC#1236) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5a10, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:11:15.676 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1236 [ CSSD]2013-05-23 18:11:15.683 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:15.683 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:15.683 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d6cf0) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:15.683 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1237 (0x7f431bfe5a90) [ CSSD]2013-05-23 18:11:15.683 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5a90 (RPC#1237) tag(4d5002d) sent to node 1 [ CSSD]2013-05-23 18:11:15.683 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04d5002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:15.683 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:15.683 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5a90 (RPC#1237) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:15.683 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5a90, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:11:15.683 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1237 [ CSSD]2013-05-23 18:11:15.683 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22d6cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:11:15.886 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ef0 [ CSSD]2013-05-23 18:11:15.886 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ef0) con(0x205e3d0) skgpid ospid 26420 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:16.746 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:16.746 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:17.086 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:18.698 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:18.699 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:11:18.752 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:18.752 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:19.117 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:20.762 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:20.762 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:21.123 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:21.169 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:11:21.170 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1238 (0x7f431bfe5b10) [ CSSD]2013-05-23 18:11:21.170 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5b10 (RPC#1238) tag(4d6002d) sent to node 1 [ CSSD]2013-05-23 18:11:21.171 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:21.171 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1238 (0x7f431bfe5b10) [ CSSD]2013-05-23 18:11:21.171 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5b10 (RPC#1238) state 6, flags 0x202 [ CSSD]2013-05-23 18:11:21.171 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5b10, ret 0, client (nil) [ CSSD]2013-05-23 18:11:21.171 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1238 [ CSSD]2013-05-23 18:11:22.768 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:22.768 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:23.129 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:23.701 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:23.701 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:11:23.947 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ef0) allocated [ CSSD]2013-05-23 18:11:23.948 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:11:23.949 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ef0 - 1,2,3 [ CSSD]2013-05-23 18:11:23.949 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x205e3d0) proc(0x22d6ef0) pid(26533) version 11:1:1:4 [ CSSD]2013-05-23 18:11:23.955 [1191196992] >TRACE: clssgmRegisterClient: proc(59/0x22d6ef0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:11:23.955 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:23.955 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 new client 0x22f1d30 with con 0x2302a70, requested num -1 [ CSSD]2013-05-23 18:11:23.955 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 18:11:23.955 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1239 (0x7f431bfe5b90) [ CSSD]2013-05-23 18:11:23.955 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5b90 (RPC#1239) tag(4d7002d) sent to node 1 [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d7002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, gid 2015733, gin 1, member 0 [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3b30/SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5b90 (RPC#1239) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5b90, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:11:23.956 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1239 [ CSSD]2013-05-23 18:11:23.961 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:23.961 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 18:11:23.961 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22d6cf0) in global grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2 [ CSSD]2013-05-23 18:11:23.961 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1240 (0x7f431bfe5c10) [ CSSD]2013-05-23 18:11:23.961 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5c10 (RPC#1240) tag(4d8002d) sent to node 1 [ CSSD]2013-05-23 18:11:23.962 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04d8002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:23.962 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member 0 [ CSSD]2013-05-23 18:11:23.962 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5c10 (RPC#1240) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:23.962 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5c10, ret 0, client 0x22f1d30 member 0x22d6cf0 [ CSSD]2013-05-23 18:11:23.962 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1240 [ CSSD]2013-05-23 18:11:23.962 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.ASM.DATABASE.ASM.prod-rac02.+asm2, member number 0 (0x22d6cf0) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:11:24.070 [1191196992] >TRACE: clssgmAllocProc: (0x22f3010) allocated [ CSSD]2013-05-23 18:11:24.071 [1207982400] >TRACE: Connect request from user oracle [ CSSD]2013-05-23 18:11:24.072 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22f3010 - 1,2,3 [ CSSD]2013-05-23 18:11:24.072 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x22f3010) pid(26538) version 11:1:1:4 [ CSSD]2013-05-23 18:11:24.086 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:11:24.087 [1191196992] >TRACE: clssgmGroupState: requested group state of group prod-rac_NG, member count 2 [ CSSD]2013-05-23 18:11:24.090 [1191196992] >TRACE: clssgmDeadProc: proc 0x22f3010 [ CSSD]2013-05-23 18:11:24.090 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22f3010) con(0x22dffa0) skgpid ospid 26538 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:24.164 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ef0 [ CSSD]2013-05-23 18:11:24.164 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ef0) con(0x205e3d0) skgpid ospid 26533 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:24.774 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:24.774 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:25.160 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:25.532 [1191196992] >TRACE: clssgmExecuteClientRequest: GRPSHDRG recvd from client 1 (0x7f431c121830) [ CSSD]2013-05-23 18:11:25.532 [1191196992] >TRACE: clssgmUnregisterShared: Same group share client 1 (0x7f431c121830), grp DG_LOCAL_DATA, member 0 [ CSSD]2013-05-23 18:11:25.532 [1191196992] >TRACE: clssgmUnregisterShared: client 1 (0x7f431c121830), grp DG_LOCAL_DATA, member 0, share type 1 [ CSSD]2013-05-23 18:11:25.532 [1191196992] >TRACE: clssgmUnreferenceMember: local grock DG_LOCAL_DATA member 0 refcount is 29 [ CSSD]2013-05-23 18:11:25.532 [1191196992] >TRACE: clssgmDeadProc: proc 0x7f431c108370 [ CSSD]2013-05-23 18:11:25.532 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x7f431c108370) con(0x7f431c0e5200) skgpid 21186 ospid 21186 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:25.835 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:25.835 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 1/1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, gid 2015734, gin 1, member 0 [ CSSD]2013-05-23 18:11:25.835 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1001) node(1) grock (0x7f431c083bb0/SRVM.DATABASE.NODEAPPS.prod-rac01) [ CSSD]2013-05-23 18:11:25.835 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac01 member(0/Remote) node(1) flags 0x1001 0x1000001 [ CSSD]2013-05-23 18:11:25.843 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:25.843 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac01, member 0 [ CSSD]2013-05-23 18:11:25.843 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac01, member number 0 (0x7f431c081ae0) node number 1 state 0x0 member refcnt 1 grock type 3 [ CSSD]2013-05-23 18:11:26.796 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:26.796 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:27.166 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:28.704 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:28.704 [1157626176] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes [ CSSD]2013-05-23 18:11:28.802 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:28.802 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:29.178 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:30.854 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:30.854 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:30.916 [1191196992] >TRACE: clssgmAllocProc: (0x22d6ee0) allocated [ CSSD]2013-05-23 18:11:30.916 [1207982400] >TRACE: Connect request from user root [ CSSD]2013-05-23 18:11:30.917 [1191196992] >TRACE: clssgmClientConnectMsg: properties of cmProc 0x22d6ee0 - 1,2,3 [ CSSD]2013-05-23 18:11:30.917 [1191196992] >TRACE: clssgmClientConnectMsg: Connect from con(0x22dffa0) proc(0x22d6ee0) pid(26619) version 11:1:1:4 [ CSSD]2013-05-23 18:11:30.922 [1191196992] >TRACE: clssgmRegisterClient: proc(58/0x22d6ee0), client(1/0x22f1d30) [ CSSD]2013-05-23 18:11:30.922 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:30.922 [1191196992] >TRACE: clssgmJoinGrock: global grock SRVM.DATABASE.NODEAPPS.prod-rac02 new client 0x22f1d30 with con 0x22d6cf0, requested num -1 [ CSSD]2013-05-23 18:11:30.923 [1191196992] >TRACE: clssgmAddGrockMember: adding member to grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:30.923 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1241 (0x7f431bfe5c90) [ CSSD]2013-05-23 18:11:30.923 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5c90 (RPC#1241) tag(4d9002d) sent to node 1 [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(5) src(1) dest(65535) size(420) tag(04d9002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmHandleJoinUpdate: (src 2/1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, gid 2015735, gin 1, member 0 [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmAddMember: granted member(0) flags(0x1) node(2) grock (0x22f3a60/SRVM.DATABASE.NODEAPPS.prod-rac02) [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmQueueGrockEvent: lockName(SRVM.DATABASE.NODEAPPS.prod-rac02) type(2) count (1/1) xwaiters(0) event(1) to memberNo(0) [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmCommonAddMember: global lock grock SRVM.DATABASE.NODEAPPS.prod-rac02 member(0/Local) node(2) flags 0x1 0x1000001 [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5c90 (RPC#1241) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmAddGrockMemCmpl: rpc 0x7f431bfe5c90, ret 0, client 0x22f1d30 member 0x22f3d60 [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmAddGrockMemCmpl: sending type 6, size 540 to 0x22f1d30 [ CSSD]2013-05-23 18:11:30.923 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1241 [ CSSD]2013-05-23 18:11:30.929 [1191196992] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (0x22f1d30) [ CSSD]2013-05-23 18:11:30.929 [1191196992] >TRACE: clssgmExitGrock: client 1 (0x22f1d30), grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:30.929 [1191196992] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (0x22f3d60) in global grock SRVM.DATABASE.NODEAPPS.prod-rac02 [ CSSD]2013-05-23 18:11:30.929 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1242 (0x7f431bfe5d10) [ CSSD]2013-05-23 18:11:30.929 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5d10 (RPC#1242) tag(4da002d) sent to node 1 [ CSSD]2013-05-23 18:11:30.929 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(6) src(1) dest(65535) size(352) tag(04da002d) incarnation(174631051) [ CSSD]2013-05-23 18:11:30.929 [1104931136] >TRACE: clssgmHandleExitUpdate: (src 1) grock SRVM.DATABASE.NODEAPPS.prod-rac02, member 0 [ CSSD]2013-05-23 18:11:30.929 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5d10 (RPC#1242) state 6, flags 0x100 [ CSSD]2013-05-23 18:11:30.929 [1104931136] >TRACE: clssgmDelMemCmpl: rpc 0x7f431bfe5d10, ret 0, client 0x22f1d30 member 0x22f3d60 [ CSSD]2013-05-23 18:11:30.930 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1242 [ CSSD]2013-05-23 18:11:30.930 [1104931136] >TRACE: clssgmRemoveMember: grock SRVM.DATABASE.NODEAPPS.prod-rac02, member number 0 (0x22f3d60) node number 2 state 0x14 member refcnt 0 grock type 3 [ CSSD]2013-05-23 18:11:31.132 [1191196992] >TRACE: clssgmDeadProc: proc 0x22d6ee0 [ CSSD]2013-05-23 18:11:31.132 [1191196992] >TRACE: clssgmDestroyProc: cleaning up proc(0x22d6ee0) con(0x22dffa0) skgpid ospid 26619 with 0 clients, refcount 0 [ CSSD]2013-05-23 18:11:31.198 [1191196992] >TRACE: clssgmGroupState: requested group state of group DBHWPROD, member count 2 [ CSSD]2013-05-23 18:11:31.198 [1191196992] >TRACE: clssgmAllocateRPCIndex: allocated rpc 1243 (0x7f431bfe5d90) [ CSSD]2013-05-23 18:11:31.199 [1191196992] >TRACE: clssgmRPC: rpc 0x7f431bfe5d90 (RPC#1243) tag(4db002d) sent to node 1 [ CSSD]2013-05-23 18:11:31.199 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(112) src(1) dest(2) size(509) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:31.199 [1104931136] >TRACE: clssgmDispatchCMXMSG(): received a good RPC response for id 1243 (0x7f431bfe5d90) [ CSSD]2013-05-23 18:11:31.199 [1104931136] >TRACE: clssgmRPCDone: rpc 0x7f431bfe5d90 (RPC#1243) state 6, flags 0x202 [ CSSD]2013-05-23 18:11:31.199 [1104931136] >TRACE: clssgmMbrDataCmpl: rpc 0x7f431bfe5d90, ret 0, client (nil) [ CSSD]2013-05-23 18:11:31.199 [1104931136] >TRACE: clssgmFreeRPCIndex: freeing rpc 1243 [ CSSD]2013-05-23 18:11:31.203 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1 [ CSSD]2013-05-23 18:11:32.706 [1157626176] >TRACE: clssnmSendingThread: sending status msg to all nodes [ CSSD]2013-05-23 18:11:32.706 [1157626176] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes [ CSSD]2013-05-23 18:11:32.860 [1104931136] >TRACE: clssgmDispatchCMXMSG: msg type(13) src(1) dest(1) size(123) tag(00000000) incarnation(174631051) [ CSSD]2013-05-23 18:11:32.860 [1104931136] >TRACE: clssgmHandleDataInvalid: grock HB+ASM, member 1 node 1, birth 7 [ CSSD]2013-05-23 18:11:33.209 [1191196992] >TRACE: clssgmExecuteClientRequest: Received data update request from client (0x7f431c05d930), type 1