Fedora主机下RHEL5+Oracle11g+VirtualBox=RAC

安装前准备:笔者主机为Fedora 14,安装所需的软件:
                《红帽企业级LinuxAS版》RedhatEnterpriseLinuxASv5.4rhel-server-5.4-i386-dvd.iso
                  linux_x86_11gR1_clusterware.zip
                  oracleasm-2.6.18-164.el5-2.0.5-1.el5.i686.rpm
                  oracleasmlib-2.0.4-1.el5.i386.rpm
                  oracleasm-support-2.1.3-1.el5.i386.rpm
                  linux_11gR1_database_1013
                  足够的硬盘空间，虚拟机 硬盘选择动态扩展，大小为20G，主机预留30G的硬盘空间 
虚拟机的配置 
首先安装虚拟机软件VirtualBox，到
http://www.virtualbox.org/wiki/Downloads
下载适合自己的rpm包
笔者使用的是fedora系统，故使用如下命令安装
rpm -Uvh VirtualBox-4.0-4.0.0_69151_fedora14-1.i686.rpm
创建虚拟机，这个地球人都会，注意以下四点：

1、内存大于等于1200M否则装rac的时候会报错
2、创建新的硬盘作为虚拟机的硬盘，不要使用已有的硬盘作为虚拟机的硬盘
3、 单击虚拟机的设置选项，点击Storage，在存储树中使用“SATA控制器”作为存储控制器，这一步一定要使用“SATA控制器”，默认的IDE控制器 无法在一个控制器上添加多个硬盘。关于这一步，读者可以先删除现有的IDE控制器，然后添加SATA控制器，最后再把以创建好的虚拟机硬盘添加到控制器 中。
4、虚拟机硬盘选择动态扩展，大小为20G
4、单击虚拟机的设置选项，点击网络选项卡，启用网络连接1和网络连接2，并都使用桥接到本地的 同一块 网卡中，笔者这里都桥接到主机的eth0网卡。

安装配置第一台RHEL5 －－rac1 
安装RHEL5网上教程很多本文不再赘述。注意满足下面三点：
安装下列软件包：

          GNOME Desktop Environment
          Editors
          Graphical Internet
          Text-based Internet
          Development Libraries
          Development Tools
          Server Configuration Tools
          Administration Tools
          Base
          System Tools
          X Window System

配置 IP地址如下

          hostname: rac1.localdomain

• IP Address eth0: 192.168.2.101 (public address)

• Default Gateway eth0: 192.168.2.1 (public address)

• IP Address eth1: 192.168.0.101 (private address)

• Default Gateway eth1: none

禁用SELINUX和防火墙

安装完成后，在虚拟机上执行，SWAP分区大于等于2g

1、安装所需软件包

关于在VirtualBox上挂载iso镜像在4.0版本中似乎有bug，笔者经常不能正常识别光盘镜像中的文件，这里笔者采用先在主机配置共
享，将iso镜像中拷贝到RHEL5中再进行安装的方式进行。关于如何在fedora下设置共享，请读者自行研究，不在本文讨论范围。

# mount -o loop 《红帽企业级LinuxAS版》RedhatEnterpriseLinuxASv5.4rhel-server-5.4-i386-dvd.iso /mnt

挂载iso镜像后,执行

cd #/mnt/Server
rpm -Uvh binutils-2.*
rpm -Uvh compat-libstdc++-33* 
rpm -Uvh elfutils-libelf-0.* 
rpm -Uvh elfutils-libelf-devel-* 
rpm -Uvh gcc-4.* 
rpm -Uvh gcc-c++-4.* 
rpm -Uvh glibc-2.* 
rpm -Uvh glibc-common-2.* 
rpm -Uvh glibc-devel-2.* 
rpm -Uvh glibc-headers-2.* 
rpm -Uvh ksh-2* 
rpm -Uvh libaio-0.* 
rpm -Uvh libaio-devel-0.* 
rpm -Uvh libgcc-4.* 
rpm -Uvh libstdc++-4.*
rpm -Uvh libstdc++-devel-4.* 
rpm -Uvh make-3.* 
rpm -Uvh sysstat-7.* 
rpm -Uvh unixODBC-2.*
rpm -Uvh unixODBC-devel-2.*

2、配置主机名和ip的映射关系

修改/etc/hosts 如下

127.0.0.1       localhost.localdomain   localhost
# Public
192.168.2.101   rac1.localdomain        rac1
192.168.2.102   rac2.localdomain        rac2
# Private
192.168.0.101   rac1-priv.localdomain   rac1-priv
192.168.0.102   rac2-priv.localdomain   rac2-priv
# Virtual
192.168.2.111   rac1-vip.localdomain    rac1-vip
192.168.2.112   rac2-vip.localdomain    rac2-vip
# SCAN
192.168.2.201   rac-scan.localdomain rac-scan

SCAN 的ip 在本次rac 安装中并不会用到，但是处于习惯笔者还是进行了定义

3、配置内核参数

修改/etc/sysctl 如下

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 1054504960
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048586

执行 [root@czmmiao ~]# /sbin/sysctl -p

4、修改shell 限制

修改?etc/security/limits.conf

oracle               soft    nproc   2047
oracle               hard    nproc   16384
oracle               soft    nofile  1024
oracle               hard    nofile  65536

5、修改登陆限制

/etc/pam.d/login

session required pam_limits.so

6、关闭selinux

修改/etc/selinux/config 

SELINUX=disabled

7、关闭时间同步服务

service ntpd stop 
# mv /etc/ntp.conf /etc/ntp.conf.orig 
# rm /var/run/ntpd.pid

8、创建必要的组、用户和目录

groupadd -g 1000 oinstall 
groupadd -g 1200 dba useradd -u 1100 -g oinstall -G dba oracle 
passwd oracle
mkdir -p /u01/app/11.2.0/grid 
mkdir -p /u01/app/oracle/product/11.2.0/db_1 
chown -R oracle:oinstall /u01 chmod -R 775 /u01/

9、在/boot/grub/grub.conf增加"divider=10"以减少空闲CPU的加载（这步为可选，不做也可以）

default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Enterprise Linux (2.6.18-194.el5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet divider=10
        initrd /initrd-2.6.18-194.el5.img

10、修改oracle的用户的环境变量

以oracle 用户登录

执行vim .bash_profile

TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_HOSTNAME=rac1.localdomain; export ORACLE_HOSTNAME
ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME
ORACLE_SID=RAC1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

if [ $USER = "oracle" ]; then
  if [ $SHELL = "/bin/ksh" ]; then
    ulimit -p 16384
    ulimit -n 65536
  else
    ulimit -u 16384 -n 65536
  fi
fi

11、以root用户安装VirtualBox的增强功能

单击Devices > Install Guest Additions菜单选项
执行

＃cd /media/VBOXADDITIONS_4.0.0_69151/




＃cp -rfp * /




＃./VBoxLinuxAdditions.run




执行init 0关机

12、为rac1增加硬盘




在主机上执行,切忌要以之前创建rac1的用户执行，例如笔者之前创建rac1的用户为czmmiao，现在也应以czmmiao执行下列命令。
$ # Create the disks and associate them with VirtualBox as virtual media.
$ VBoxManage createhd --filename asm1.vdi --size 1024 --format VDI --variant Fixed
$ VBoxManage createhd --filename asm2.vdi --size 1024 --format VDI --variant Fixed
$ VBoxManage createhd --filename asm3.vdi --size 1024 --format VDI --variant Fixed
$
$ # Connect them to the VM.
$ VBoxManage storageattach rac1 --storagectl "SATA Controller" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
eable
$ VBoxManage storageattach rac1 --storagectl "SATA Controller" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
pe shareable
$ VBoxManage storageattach rac1 --storagectl "SATA Controller" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
pe shareable 
$
$ # Make shareable.
$ VBoxManage modifyhd asm1.vdi --type shareable
$ VBoxManage modifyhd asm2.vdi --type shareable
$ VBoxManage modifyhd asm3.vdi --type shareable




13、重新开启rac1，对硬盘进行分区
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305):
Using default value 1305

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1305    10482381   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#
分区后的结果如下所示




[root@rac1 ~]# ls /dev/sd*
/dev/sda   /dev/sda2  /dev/sdb1  /dev/sdc   /dev/sdd
/dev/sda1  /dev/sdb   /dev/sdb2  /dev/sdc1  /dev/sdd1





13、安装配置ASM管理软件和Cluster管理软件
[root@rac1 ~]# uname -r
2.6.18-164.el5




到Oracle官网上下载
oracleasm-2.6.18-164.el5-2.0.5-1.el5.i686.rpm
oracleasmlib-2.0.4-1.el5.i386.rpm
oracleasm-support-2.1.3-1.el5.i386.rpm
linux_x86_11gR1_clusterware.zip

＃rpm -Uvh oracleasm*.rpm




cd /your/path/to/grid/rpm
rpm -Uvh cvuqdisk*





配置ASM
# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: 
Writing Oracle ASM library driver configuration: done
#





装载内核
# /usr/sbin/oracleasm init
Loading module "oracleasm": oracleasm
Mounting ASMlib driver filesystem: /dev/oracleasm
# 





创建ASM文件系统




# /usr/sbin/oracleasm createdisk VOL1 /dev/sdc1
Writing disk header: done
Instantiating disk: done
# /usr/sbin/oracleasm createdisk VOL2 /dev/sdd1
Writing disk header: done
Instantiating disk: done




# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...












14、创建裸设备
#vim /etc/sysconfig/rawdevices
/dev/raw/raw1   /dev/sdb1
/dev/raw/raw2   /dev/sdb2
#vim /etc/udev/rules.d/50-udev.rules
KERNEL=="raw[0-9]*",MODE="0660",GROUP="oinstall",OWNER="oracle",NAME="raw/%k"




#shutdown -h now




至此第一台虚拟机rac1配置完成，笔者以汗流浃背

创建第二台虚拟机--rac2




在主机上执行
$ VBoxManage clonehd rac1.vdi rac2.vdi




注意，这里的虚拟机硬盘选择“以存在的虚拟机硬盘”选择
rac2.vdi




$ VBoxManage storageattach rac2 --storagectl "SATA Controller" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
eable
$ VBoxManage storageattach rac2 --storagectl "SATA Controller" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
pe shareable
$ VBoxManage storageattach rac2 --storagectl "SATA Controller" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
pe shareable 






	
hostname: rac2.localdomain












配置rac2网络如下

IP   Address eth0: 192.168.2.102 (public address) 




Default  Gateway eth0: 192.168.2.1 (public address) 




IP   Address eth1: 192.168.0.102 (private address) 




Default Gateway eth1:none 

 
更改oracle用户环境变量
以oracle用户登录 
$vim .bash_profile
ORACLE_SID=RAC2; export ORACLE_SID
ORACLE_HOSTNAME=rac2.localdomain; export ORACLE_HOSTNAME 

配置rac1和rac2的等价性 
以oracle用户登录rac1的Xwindows， 
mkdir   .ssh
chmod 700 .ssh
ssh-keygen -t rsa
ssh-keygen -t dsa
cat *.pub > authorized_keys
scp  authorized_keys rac2:/home/oracle/.ssh/keys_dbs 
在rac2上执行 
mkdir   .ssh
chmod 700 .ssh
ssh-keygen -t rsa
ssh-keygen -t dsa
cat *.pub > authorized_keys
cat  keys_dbs >> authorized_keys
scp  authorized_keys rac1:/home/oracle/.ssh/ 
在rac1和rac2分别执行
ssh rac1 ssh rac1-priv ssh rac2 ssh rac2-priv

安装Clusterware群集管理软件




安装前测试
/mountpoint/clusterware/runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose

确保rac1和rac2处于开机状态，以oracle用户登录rac1
su - root
/etc/init.d/rawdevices start
exit 
./runInstall 
进行安装，进行到

 点击“添加”

 继续进行至

 进行至
Specify Oracle Cluster Register (OCR) Location 时，
选择
External Redundancy 
/dev/raw/raw1 
进行至
Specify Voting Disk Location时 
选择
External Redundancy
/dev/raw/raw2
点击下一步、安装之至：出现提示框
在两个节点上分别执行提示中的脚本，切忌，不能在两个节点上同时执行，完成后点击OK，安装成功。
安装Oracle Database
注意一下两点即可：
选择企业版安装
选择之安装
一路回车到最后遇到提示框时，在两个节点上分别执行提示中的脚本即可
至此，大公告成