Monday, May 9, 2016

The ps Command

The ps (i.e., process status) command is used to provide information about the currently running processes, including their process identification numbers (PIDs).

Show all Server processes 
ps -ef |grep oracledcpdb1    --here, dcpdb1 is instance name 

Total count of server processes 
ps -ef |grep oracledcpdb1 |grep -v grep |wc -l   ---here, dcpdb1 is instance name

Show background processes 
ps -ef |grep dcpdb1  --here, dcpdb1 is instance name


ORA-01110: data file 6: '/areadb/datafile/credittestdata01.dbf'

Error Raise
SQL> shutdown immediate
ORA-01116: error in opening database file 6
ORA-01110: data file 6: '/areadb/datafile/credittestdata01.dbf'
ORA-27041: unable to open file
HPUX-ia64 Error: 2: No such file or directory
Additional information: 3
SQL>
SQL>

Solution
SQL> alter database datafile 6 offline drop;

Database altered.

SQL>
SQL>shutdown immediate


ORA-16191: Primary log shipping client not logged on standby

Reasons: 
Primary and standby database SYS password is not same and it creates conflicts to access the standby database and vice versa.

Solution:

Step-1:  Check error and remote_login_passwordfile parameter

SQL> SELECT DESTINATION, STATUS, ERROR FROM V$ARCHIVE_DEST WHERE DEST_ID=2;

DESTINATION
--------------------------------------------------------------------------------
STATUS    ERROR
--------- -----------------------------------------------------------------
dcstdb
ERROR     ORA-16191: Primary log shipping client not logged on standby


SQL> show parameter remote_login_passwordfile;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
remote_login_passwordfile            string      EXCLUSIVE
SQL>


Step-2: Disable log_archive_dest_state_2

SQL> show parameter log_archive_dest_state_2;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_state_2             string      ENABLE
log_archive_dest_state_20            string      enable
log_archive_dest_state_21            string      enable
log_archive_dest_state_22            string      enable
log_archive_dest_state_23            string      enable
log_archive_dest_state_24            string      enable
log_archive_dest_state_25            string      enable
log_archive_dest_state_26            string      enable
log_archive_dest_state_27            string      enable
log_archive_dest_state_28            string      enable
log_archive_dest_state_29            string      enable

SQL>
SQL> alter system set log_archive_dest_state_2=DEFER sid='*' scope=both;

System altered.

SQL> show parameter log_archive_dest_state_2;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_state_2             string      DEFER
log_archive_dest_state_20            string      enable
log_archive_dest_state_21            string      enable
log_archive_dest_state_22            string      enable
log_archive_dest_state_23            string      enable
log_archive_dest_state_24            string      enable
log_archive_dest_state_25            string      enable
log_archive_dest_state_26            string      enable
log_archive_dest_state_27            string      enable
log_archive_dest_state_28            string      enable
log_archive_dest_state_29            string      enable
SQL> 


Step-3: Re-Create password file by following command

oracle@testbk [/u02/app/oracle/product/11.2.0.4/dbhome_1/dbs]$orapwd FILE=orapw$ORACLE_SID ENTRIES=30 ignorecase=Y
Enter password for SYS:


Step-4: Shutdown Standby Server

oracle@dcstdb [/u02/app/oracle/product/11.2.0.4/dbhome_1/dbs]$sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Tue May 3 16:43:01 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.


Step-5: Transfer Primary DB password file to Standby server
oracle@testbk [/u02/app/oracle/product/11.2.0.4/dbhome_1/dbs]$scp orapwtestbk1 oracle@dcstdb:/u02/app/oracle/product/11.2.0.4/dbhome_1/dbs/orapwdcstdb1                                
Password:
orapwtestbk1                                                          100% 6144     6.0KB/s   6.0KB/s   00:00


Step-6: Startup Standby Server

SQL> startup
ORACLE instance started.

Total System Global Area 5127602176 bytes
Fixed Size                  2214432 bytes
Variable Size            4580181472 bytes
Database Buffers          536870912 bytes
Redo Buffers                8335360 bytes
Database mounted.
Database opened.
SQL>

Step-7: recovery process start

SQL> alter database reocver managed standby database using current logfile disconnect from session;


Step-8: Enable log_archive_dest_state_2 of PRIMARY.

SQL> alter system set log_archive_dest_state_2=ENABLE sid='*' scope=both;
SQL> alter system switch all logfile;

SQL> SELECT DESTINATION, STATUS, ERROR FROM V$ARCHIVE_DEST WHERE DEST_ID=2;

DESTINATION
--------------------------------------------------------------------------------
STATUS    ERROR
--------- -----------------------------------------------------------------
dcstdb
VALID

Create PFILE of DataGuard (Standby Server)

For DataGuard, pfile creation is a important step, following are a standby server pfile example:

Primary Server: testbk
Standby Server: dcstdb

*.audit_file_dest='/u02/app/oracle/admin/dcstdb/adump'
*.audit_trail='db'
*.compatible='11.2.0.4.0'
*.control_files='+REDO/dcstdb/controlfile/current.256.910371717','+CONTROL/dcstdb/controlfile/current.256.910371719'
*.db_block_size=8192
*.db_create_file_dest='+DATA'
*.db_create_online_log_dest_1='+REDO'
*.db_create_online_log_dest_2='+CONTROL'
*.db_domain='nblitd.com'
*.db_file_name_convert='TESTBK','DCSTDB'
*.db_name='testbk'
*.db_recovery_file_dest='+FRA'
*.db_recovery_file_dest_size=85899345920
*.db_unique_name='dcstdb'
*.diagnostic_dest='/u02/app/oracle'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=dcpdbXDB)'
*.fal_client='DCSTDB'
*.fal_server='TESTBK'
*.filesystemio_options='SETALL'
testbk1.instance_number=1
*.log_archive_config='DG_CONFIG=(testbk,dcstdb)'
*.log_archive_dest_1='SERVICE=testbk NOAFFIRM ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=testbk'
*.log_archive_dest_2='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=dcstdb'
*.log_archive_dest_state_1='ENABLE'
*.log_archive_dest_state_2='ENABLE'
*.log_archive_format='%t_%s_%r.arc'
*.log_archive_max_processes=10
*.log_file_name_convert='TESTBK','DCSTDB'
*.memory_target=8589934592
*.open_cursors=5000
*.processes=5000
*.remote_login_passwordfile='exclusive'
*.session_cached_cursors=5000
*.sessions=7524
*.standby_file_management='AUTO'
testbk1.thread=1
testbk1.undo_tablespace='UNDOTBS1'

Sunday, May 8, 2016

Dataguard SWITCHOVER on Oracle 11gr2 RAC Database (Using Physical Standby)

A primary RAC database that is running normally, the switchover operation must start from the primary site. Below are the step of switchover operation:

a. Shutdown all primary instances except one.
b. Shutdown all standby instances except one.
c. On the Primary site
    i. SQL> select switchover_status from V$database;
    ii. If switchover_status shows ‘sessions active’ go to step iii. If switchover_status shows      
        ‘TO_STANDBY’ go to step v.
    iii.SQL> select sid, process, program from v$session where type='user' and sid <>(select distinct 
         sid from V$mystat);
    iv. if any rows/process found then kill those process or wait until logout. 
    v. SQL> alter database commit to switchover to physical standby with session shutdown;
    vi. SQL>shutdown immediate;
    vii. SQL>exit;
    viii. Restart the instance and mount it as standby
                        SQL> startup nomount;
                        SQL> alter database mount standby database;
    ix.   Open database in read only mode
                        SQL> alter database mount standby database;
    x.   Start recover process
          SQL> alter database recover managed standby database using current logfile disconnect from 
                     session;

d. On the Standby site.
     i. SQL>alter database recover managed standby database cancel;
    ii. SQL>alter database recover managed standby database nodelay disconnect from session          
         through last switchover;
          Note: Above (ii)  is need only when a time lag used.
   iii. SQL> alter database commit to switchover to primary with session shutdown;
   iv.  SQL> alter database open;

e. On the original primary site mount & open the other instances as standby.
f. On the original DR site start up and open the other instances (as primary).
g. Verify the new data guard configuration
     i. On new Primary
        SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;
    ii. On new standby
         make sure the log file is copied over and applied or will be applied (if time lag in apply is used).


IMPLEMENTATION
Primary server : 172.xx.xxx.60 
Standby Server : 172.xx.xxx.45

Step-1: On Primary Server (172.xx.xxx.60) 
SQL> select switchover_status from v$database;

SWITCHOVER_STATUS
--------------------
SESSIONS ACTIVE

SQL> select database_role, open_mode from v$database;

DATABASE_ROLE    OPEN_MODE
---------------- --------------------
PRIMARY          READ WRITE

SQL> select sid,process,program from v$session where type='user';

no rows selected


Step-2: On Standby Server (172.xx.xxx.45)

SQL> select switchover_status from v$database;

SWITCHOVER_STATUS
--------------------
NOT ALLOWED

SQL> select database_role, open_mode from v$database;

DATABASE_ROLE    OPEN_MODE
---------------- --------------------
PHYSICAL STANDBY READ ONLY WITH APPLY


Step-3: On Primary Server (172.xx.xxx.60)

SQL> alter database commit to switchover to physical standby with session shutdown;
Database altered.

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
oracle@testbk [/home/oracle]$sqlplus / as sysdba;

SQL*Plus: Release 11.2.0.4.0 Production on Thu May 5 16:59:02 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 5127602176 bytes
Fixed Size                  2214432 bytes
Variable Size            4580181472 bytes
Database Buffers          536870912 bytes
Redo Buffers                8335360 bytes
SQL>


Step-4: On Standby (172.xx.xxx.45)

SQL>  select switchover_status from v$database;

SWITCHOVER_STATUS
--------------------
SESSIONS ACTIVE


Step-6: On Primary (172.xx.xxx.60)

SQL> alter database mount standby database;

Database altered.


SQL> alter database recover managed standby database disconnect;

Database altered.


Step-7: On Standby (172.xx.xxx.45)

SQL> alter database recover managed standby database cancel;

Database altered.

SQL> select database_role, open_mode from v$database;

DATABASE_ROLE    OPEN_MODE
---------------- --------------------
PHYSICAL STANDBY READ ONLY

SQL> alter database recover managed standby database nodelay disconnect from session through last switchover;
alter database recover managed standby database nodelay disconnect from session through last switchover
*
ERROR at line 1:
ORA-16177: media recovery is not required


SQL> alter database commit to switchover to primary with session shutdown;

Database altered.

SQL> select database_role, open_mode from v$database;

DATABASE_ROLE    OPEN_MODE
---------------- --------------------
PRIMARY          MOUNTED


SQL>  alter database open;

Database altered.

SQL> select database_role, open_mode from v$database;

DATABASE_ROLE    OPEN_MODE
---------------- --------------------
PRIMARY          READ WRITE


Step-8: On New Primary (172.xx.xxx.45)

SQL>  alter system archive log current;

System altered.

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     196
Next log sequence to archive   199
Current log sequence           199

SQL> alter system switch logfile;

System altered.

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     196
Next log sequence to archive   200
Current log sequence           200
SQL>


Step-9: On New standby (172.xx.xxx.60)
SQL> archive log list
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     0
Next log sequence to archive   0
Current log sequence           199
SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     0
Next log sequence to archive   0


Current log sequence           200