Step by Step - Configure Direct NFS Client (DNFS) on Linux for Oracle11g:
1 Shutdown the database
sqlplus "/as sysdba"
shutdown immediate
exit
2 Enable the Direct NFS Client
cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk dnfs_on
1 Shutdown the database
sqlplus "/as sysdba"
shutdown immediate
exit
2 Enable the Direct NFS Client
cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk dnfs_on
Verify:
ls -l $ORACLE_HOME/lib/lib*od*11.so
-rw-r--r--. 1 oracle oinstall 60431 Sep 17 2011 $OH/lib/libnfsodm11.so << dnfs
-rw-r--r--. 1 oracle oinstall 60431 Oct 20 19:45 $OH/lib/libodm11.so # Oracle used
-rw-r--r--. 1 oracle oinstall 12259 Sep 17 2011 $OH/lib/libodmd11.so << no dnfs
( Disable the Direct NFS Client
cd $ORACLE_HOME/rdbms/lib
make –f ins_rdbms.mk dnfs_off )
Oracle processes use /etc/mtab to access NFS settings
3 Configure ORANFSTAB
Direct NFS Client can use a new configuration file or the mount tab file (/etc/mtab on Linux) to determine the mount point settings for NFS storage devices.
This file is required only for configuring the Direct NFS for load balancing and specfic to single database.You can still enable the Direct NFS without configuring oranfstab file.DNFS will take mount point settings for NFS from /etc/mtab on Linux
In RAC,the oranfstab must be configured on all nodes and keep /etc/oranfstab file synchronized on all nodes.
(When the oranfstab file is placed in $ORACLE_HOME/dbs, the entries in the file are specific to a single database. In this case, all nodes running an Oracle RAC database use the same ORACLE_HOME/dbs/oranfstab file.
When the oranfstab file is placed in /etc, then it is globally available to all Oracle databases, and can contain mount points used by all Oracle databases running on nodes in the cluster, including single-instance databases. However, on Oracle RAC systems, if the oranfstab file is placed in /etc, then you must replicate the file /etc/oranfstab file on all nodes, and keep each /etc/oranfstab file synchronized on all nodes, just as you must with the /etc/fstab file.
cd /etc
grant root/root access to oranfstab
update oranfstab file with /oradata and /orabkup mount point
Example file provided below for MYDBDEV:
$ORACLE_HOME/dbs/oranfstab or cat /etc/oranfstab
server: stgasm <=== NFS server Host name
path: 10.177.52.158 <--- First path to NFS server ie NFS server NIC
local: 10.177.52.151 <--- First client-side NIC
path: 10.177.52.159 <--- Second path to NFS server ie NFS server NIC (For load balance purpose)
local: 10.177.52.151 <--- Second client-side NIC (For load balance purpose)
export: /oraclenfs mount: /oradata1
or
server: (Storage server Name)
path: <IP 1.1.1.1>
#path:
#path:
#path:
export: /vol/<XXXXXXX>: /<MP>
server: (Storage server Name)
path: <IP 1.1.1.1>
#path:
#path:
#path:
export: /vol/<XXXXXXX>: /<MP>
dontroute
4 Verify mtab
cd /etc
Verify mount point entries on mtab for which you want to setup the DNFS.
Example: MYDBDEV
<Storage host name >:/vol/mydbdev_MP /MP nfs rw,nolock,bg,hard,nointr,rsize=32768,wsize=32768,tcp,nfsvers=3,timeo=600,addr=10.1.1.1 0 0
<Storage host name >:/vol/mydbdev_MP /MP nfs rw,nolock,bg,hard,nointr,rsize=32768,wsize=32768,tcp,nfsvers=3,timeo=600,addr=10.1.1.1 0 0
5 . What is oradism? When is it used by Direct NFS?
Direct NFS client uses oradism executable to perform functions that require root privileges. oradism is configured by default on a single instance and for RAC if not using shared $ORACLE_HOME.
It is used for the following special purposes:
Direct NFS client needs root privileges to start communication with the NFS filer. It uses oradism to obtain the root file handle for the exported volume and the NFS server port and NFS mount port. Once the root handle and port information is obtained, all future communication is issued by Oracle user processes using normal privileges.
Oradism executable need always to be owned by root otherwise instance will not come up and returns the error in alert log:
Direct NFS: please check that oradism is setuid
Verify oradism
cd $ORACLE_HOME/bin
verify the permissions on oradism binary. It should have root oradba grants
If the binary doesn't have the right grants: Follow the steps below
# chown root:oradba oradism / chmod 4755 oradism /cd $ORACLE_HOME/bin/oradism
If ‘oradism’ is not configured correctly, this error in seen the alert log:
“Direct NFS: please check that oradism is setuid”
6. Restart the database server
sudo reboot –p
7. Verify the status of the database
If it's not started. Syntax: sqlplus "/as sysdba" , startup
How can the DNFS usage be verified?
1. Verify the status of DNFS
Vi my_db_dnfs_layout.sql
spool db_dnfs_layout.html
SET MARKUP HTML ON
set echo on
set pagesize 200
alter session set nls_date_format='DD-MON-YYYY HH24:MI:SS';
select 'THIS DB REPORT WAS GENERATED AT: ==)> ' , sysdate " " from dual;
select 'HOSTNAME ASSOCIATED WITH THIS DB INSTANCE: ==)> ' , MACHINE " " from v$session where program like '%SMON%';
select * from v$dnfs_servers;
select * from v$dnfs_files;
select * from v$dnfs_channels;
select * from v$dnfs_stats;
select * from v$datafile;
select * from dba_data_files;
select * from DBA_TEMP_FILES;
select * from v$logfile;
select * from v$log;
select * from v$version;
show parameter
spool off
exit
sqlplus "/as sysdba"
@ my_db_dnfs_layout.sql
2. Just after the initialization parameters are listed in the alert log, you will see the following entry
Direct NFS client is enabled by checking the below message
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version2.0
3. Along with the message from the alert log, this query on v$dnfs_servers ensures that dNFS is truly being used (returns !=0 value):
select count(*) from v$dnfs_servers
( sometimes it isn’t true ;)
4. $ ps -ef | grep dbw
oracle 23126 1 0 06:53 ? 00:00:00 ora_dbw0_MYINST1
oracle 23128 1 0 06:53 ? 00:00:00 ora_dbw1_MYINST1
oracle 23246 22377 0 07:04 pts/0 00:00:00 grep dbw
$ lsof –p < dbw pid >
/usr/sbin/lsof -p 23126
# DNFS OFF
lsof -p 725 | grep data01.dbf
oracle 725 oracle 262u REG ... /nfsimp/data01.dbf (192.168.51.21:/u01)
# DNFS ON
lsof -p 6540 | grep 192.168.51.21
oracle 6540 oracle 32u IPv4 ... TCP dbhost:26171->nfsserver:nfs (ESTABLISHED)
5. alert.log
Direct NFS: channel id [0] path [IPnfs] to filer [KUKARACHA] via local [IPdb] is UP
Direct NFS: channel id [1] path [IPnfs] to filer [KUKARACHA] via local [IPdb] is UP
About DNFS things to remember for knowledge purpose::
1 DNFS oranfstab
Parameter
|
Usage
|
Server
|
Unique
identifier for the NFS server
|
Local
|
Network
paths (up to 4) on the database host
|
Path
|
Network
paths (up to 4) on the NFS server
|
Export
|
The
exported volume on the NFS server
|
Mount
|
The
local mount point for the exported volume
|
mnt_timeout
|
Time
in seconds to wait for the first mount
|
dontroute
|
Prevents
OS routing of outgoing messages
|
management
|
Network
path for NFS server management interface
|
2 Direct NFS searches for mount entries in the following order
$ORACLE_HOME/dbs/oranfstab # an instance specific
/etc/oranfstab
/etc/mtab
3 You must restart an instance after adjusting oranfstab file
Disadvantages of DNFS ::
· It is relatively difficult to troubleshoot
· You will not find many DNFS messages in alert.log or trace files
· Sometimes Oracle SILINTLY switches from dNFS to kNFS
· This sections gives you useful troubleshooting hints
VVI Note::
Don’t mix old and new methods
● Old method
cd $ORACLE_HOME/lib
mv libodm11.so libodm11.so_bak
ln -s libnfsodm11.so libodm11.so
● New/current method
make -f ins_rdbms.mk dnfs_on
● If you enable dNFS using one method disable it the same way.
