ASGAUR

If you suspect that hardware problems are the reason for errors in your installation, analyze the system log. To do this, enter the command:
errpt -a | more

1. Create volume group.
See the section Creating Volume Groups.
2. Create logical volume:
a. Enter the command smitty.
b. Select:

Physical & Logical Storage → Logical Volume Manager → Logical Volumes → Add a Logical Volume

c. Enter volume group name.
d. Enter logical volume name (for example: lvsap02).
e. Enter the number of logical partitions.
f. Set the logical volume TYPE to r3data and press ENTER.

g. Press F10 to exit SMIT.
The logical volume can be accessed through /dev/r<name of raw
log.vol>.
Name of logical volume: lvsap02 leads to the corresponding device name
/dev/rlvsap02.
3. Accessing raw devices
For each logical volume, create a link to access the volume.
DB2 UDB for UNIX and Windows: Not necessary
Informix: ln -s /dev/rlvsap01 \
/informix/<SAPSID>/sapdata/physdev1/data1
Oracle: Not necessary

SAP DB: Not necessary (R3SETUP creates the links)

1. Create one logical volume for each file system listed in the your profile.
a. Select:
Physical & Logical Storage → Logical Volume Manager → Logical Volumes → Add a Logical Volume
b. Enter a volume group name, for example, sapr3vg.
c. Enter a logical volume name, for example, lvsap01.
d. Enter the number of logical partitions (see section Size of a Logical Partition).
e. Press F3 until the Physical & Logical Storage menu appears.
2. Create the file systems.
a. Select:
Physical & Logical Storage → File Systems → Add/Change/Show/Delete File Systems → Journaled File Systems →
Add a Journaled File System on a previously defined Logical Volume
b. Press F4 to get a list of logical volumes.
c. Select one logical volume.
Select the logical volume with the desired size.
Enter mount point as given in the file system list files.
DB2 UDB for UNIX and Windows: /db2/<SAPSDID>
Informix: /informix/<SAPSID>
Oracle: /oracle/<SAPSID>/sapdata1
SAP DB: /sapdb/<SAPSID>/sapdata
Set mount automatically to yes.
d. Press F10 to exit smitty.

Cluster Topology Configuration:

NOTE: Underscore ( _ ) is the only valid special character for HACMP objects names
First character must not be numeric.

From one node, you will execute the following steps
# smit hacmp
Extended Configuration → Extended Topology Configuration →  Configure Cluster → Configure Nodes →  Configure IP Networks

• Configure Communication Interfaces on these IP networks
• Configure non-IP Networks (serial rs232 or disk heartbeat)
• Define Communication Devices on these non-IP networks
• Configure persistent IP addresses for network(s) with boot or service
• Synchronize configuration.    It will also setup persistent IP aliases.

eg.
Configure HACMP Networks

Move cursor to desired item and press Enter.

Add a Network to the HACMP Cluster
Change/Show a Network in the HACMP Cluster
Remove a Network from the HACMP Cluster

+----------------------------------------------------------------------
|                     Select a Network to Change/Show
|
| Move cursor to desired item and press Enter.
|
|   net_ether_01 (10.0.51.0/24 172.16.1.0/24 172.17.1.0/24)
|   net_rs232_01


Change/Show an IP-Based Network in the HACMP Cluster

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
* Network Name                                        net_ether_01
New Network Name                                   []
* Network Type                                       [ether]                 +
* Netmask                                            [255.255.255.0]         +
* Enable IP Address Takeover via IP Aliases          [Yes]                   +
IP Address Offset for Heartbeating over IP Aliases []
* Network attribute                                   public                 +

Change/Show a Serial Network in the HACMP Cluster

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
* Network Name                                        net_rs232_01
New Network Name                                   []
* Network Type                                       [rs232]                 +



Configure HACMP Communication Interfaces/Devices

Mo+--------------------------------------------------------------------------+
|          Select a Communication Interface/Device to Change/Show          |
|                                                                          |
| Move cursor to desired item and press Enter. Use arrow keys to scroll.   |
|                                                                          |
| [TOP]                                                                    |
|   # Node / Network                                                       |
|   #       Interface/Device  IP Label/Device Path              IP Address |
|                                                                          |
|                                                                          |
|   # RACNODE1 / net_ether_01                                              |
|           en0               ibmcl01_boot                      172.16.1.1 |
|           en2               ibmcl01_stby                      172.17.1.1 |
|                                                                          |
|   # RACNODE1 / net_rs232_01                                              |
|           tty0              RACNODE1_tty0_01                  /dev/tty0  |
|                                                                          |
|                                                                          |
|   # RACNODE2 / net_ether_01                                              |
|           en0               ibmcl02_boot                      172.16.1.1 |
|           en2               ibmcl02_stby                      172.17.1.1 |
|                                                                          |
|   # RACNODE2 / net_rs232_01                                              |
|           tty0              RACNODE2_tty0_01                  /dev/tty0  |
| [BOTTOM]                                                                 |

1) Nodes prerequisites

Network configuration on EACH node interfaces on their “boot”  and  “standby” address, each one on a different subnet hostname corresponding to future “persistent” address:

# smit mkhostname

• /etc/hosts must contain all nodes addresses, persistent and application IP aliases.
  
• /.rhosts should be empty, except for Bull ClusterWatch (corrected in future PTF).
• on each node file  /usr/es/sbin/cluster/etc/rhosts must contain all the “boot” addresses of all nodes and all HACMP networks, for example:

clusnod1_boot1
clusnod2_boot1

Check also that the following daemon is running :
# lssrc –g clcomdES
clcomdES         clcomdES         13350   active
otherwise start it with     startsrsc –s clcomdES

• godm TCP/IP service must be active ( refresh –s inetd  on all nodes)
• Volume Groups imported on all nodes, not automatically activable at boot time,
  

If AIX 5.2, they must be concurrent capable (bos.clvm.enh fileset installed)
Remove Installed Software

Ty+--------------------------------------------------------------------------+
Pr|                              SOFTWARE name                               |
|                                                                          |
| Move cursor to desired item and press Esc+7.                             |
* |     ONE OR MORE items can be selected.                                   |
| Press Enter AFTER making all selections.                                 |
|                                                                          |
| [MORE...580]                                                             |
|   rpm.rte                                                                |
| > rsct.basic.hacmp                                                       |
| > rsct.basic.rte                                                         |
| > rsct.compat.basic.hacmp                                                |
| > rsct.compat.basic.rte                                                  |
| > rsct.compat.clients.hacmp                                              |
| > rsct.compat.clients.rte                                                |
| > rsct.core.auditrm                                                      |
| > rsct.core.errm                                                         |
| > rsct.core.fsrm                                                         |
| > rsct.core.gui                                                          |
| > rsct.core.hostrm                                                       |
| > rsct.core.lprm                                                         |
| > rsct.core.rmc                                                          |
| > rsct.core.sec                                                          |
| > rsct.core.sensorrm                                                     |
| > rsct.core.sr                                                           |
| > rsct.core.utils                                                        |
| > rsct.msg.EN_US.core.auditrm                                            |
| > rsct.msg.EN_US.core.errm                                               |
| > rsct.msg.EN_US.core.fsrm                                               |
| > rsct.msg.EN_US.core.gui                                                |
| > rsct.msg.EN_US.core.hostrm                                             |
| > rsct.msg.EN_US.core.lprm                                               |
| > rsct.msg.EN_US.core.rmc                                                |
| > rsct.msg.EN_US.core.sec                                                |
| > rsct.msg.EN_US.core.sensorrm                                           |
| > rsct.msg.EN_US.core.sr                                                 |
| > rsct.msg.EN_US.core.utils                                              |
| > rsct.msg.en_US.basic.rte                                               |
| > rsct.msg.en_US.core.auditrm                                            |
| > rsct.msg.en_US.core.errm                                               |
| > rsct.msg.en_US.core.fsrm                                               |
| > rsct.msg.en_US.core.gui                                                |
| > rsct.msg.en_US.core.gui.com                                            |
| > rsct.msg.en_US.core.hostrm                                             |
| > rsct.msg.en_US.core.lprm                                               |
| > rsct.msg.en_US.core.rmc                                                |
| > rsct.msg.en_US.core.rmc.com                                            |
| > rsct.msg.en_US.core.sec                                                |
| > rsct.msg.en_US.core.sensorrm                                           |
| > rsct.msg.en_US.core.sr                                                 |
| > rsct.msg.en_US.core.utils                                              |
| > rsct.opt.storagerm
| > cluster.adt.es.client.include                                          |
| > cluster.adt.es.client.samples.clinfo                                   |
| > cluster.adt.es.client.samples.clstat                                   |
| > cluster.adt.es.client.samples.libcl                                    |
| > cluster.adt.es.java.demo.monitor                                       |
| > cluster.doc.en_US.es.html                                              |
| > cluster.doc.en_US.es.pdf                                               |
| > cluster.es.cfs.rte                                                     |
| > cluster.es.client.lib
| > cluster.es.client.rte                                                  |
| > cluster.es.client.utils                                                |
| > cluster.es.client.wsm                                                  |
| > cluster.es.cspoc.cmds                                                  |
| > cluster.es.cspoc.dsh                                                   |
| > cluster.es.cspoc.rte                                                   |
| > cluster.es.plugins.dhcp                                                |
| > cluster.es.plugins.dns                                                 |
| > cluster.es.plugins.printserver
| > cluster.es.server.cfgast                                               |
| > cluster.es.server.diag                                                 |
| > cluster.es.server.events                                               |
| > cluster.es.server.rte                                                  |
| > cluster.es.server.testtool                                             |
| > cluster.es.server.utils                                                |
| > cluster.es.worksheets                                                  |
| > cluster.license                                                        |
| > cluster.man.en_US.es.data
| > cluster.msg.en_US.cspoc                                                |
| > cluster.msg.en_US.es.client                                            |
| > cluster.msg.en_US.es.server                                            |                                             |                                        |                                                 |                                                     |
Es+--------------------------------------------------------------------------+

1) Configure Cluster

# smit hacmp
Extended Configuration → Extended Topology Configuration → Configure an HACMP Cluster →         Add/Change/Show an HACMP Cluster

Assign a unique Cluster Name (< 32 characters)

2) Configure Nodes

# smit hacmp
Extended Configuration → Extended Topology Configuration → Configure HACMP Nodes →  Add a Node to the HACMP Cluster

Give Node Name, each node has a unique name (<32 characters)
Communication Path to Node: Press F4 and select an IP label (e.g.: boot address)
Repeat this step for all nodes.

3) Configure IP Networks

# smit hacmp
Extended Configuration → Extended Topology Configuration → Configure HACMP Networks →                Add a Network to the HACMP Cluster

Go to    # Pre-defined IP-based Network Types
select network type (e.g.: ether)
check  that Netmask is correct
It is mandatory to modify the following field:
* Enable IP Address Takeover via IP Aliases          [No]           
Repeat this step for all IP networks (interconnect, administration).        

Configure Communication Interfaces on these IP networks
# smit hacmp
Extended Configuration → Extended Topology Configuration → Configure HACMP Communication → Interfaces/Devices
Add Communication Interfaces/Devices    
Add Pre-defined Communication Interfaces and Devices
Communication Interfaces
Select a previously defined  network
give IP Label and Node Name Repeat this step for all  “boot and standby” addresses of all IP networks and of all nodes. E.g.: clusnod1_boot, clusnod1_stby, clusnod2_boot, clusnod2_stby               

4) Configure non-IP network (serial rs232 or disk heartbeat)

# smit hacmp
Extended Configuration → Extended Topology Configuration → Configure HACMP Networks → Add a Network to the HACMP Cluster
Go to   # Pre-defined Serial Device Types
Select  rs232 or diskhb
This will create a network with a default name like net_rs232_0<x>

Define Communication Devices on these non-IP networks
# smit hacmp
Extended Configuration →   Extended Topology Configuration →   Configure HACMP Communication Interfaces/Devices

Add Communication Interfaces/Devices
Add Pre-defined Communication Interfaces and Devices
Communication Devices

Select a Device Network then give:
Device Name:    symbolic name eg.  clusnod1_tty
Device Path  :    /dev/tty<x>  or    /dev/hdiskpower<x>   (or  /dev/hdisk<x&gt
disk must belong to an  AIX 5.2 enhanced concurrent VG
Node Name   :     name of one node

Repeat this step to define the Communication Device of the other node(s) …

5) Configure persistent IP addresses for network(s) where an initial  “boot” address which will be replaced by a “service” address

# smit hacmp
Extended Configuration → Extended Topology Configuration →   Configure HACMP Persistent Node IP Label / Addresses
Add a Persistent Node IP Label / Address
Select a Node and give:
* Network Name            [ ]          Press  F4
* Node IP Label / Address    [ ]          Should usually correspond to node hostname
or Press  F4 to select from /etc/hosts

Repeat this step for the other interface(s) on other node(s)  …

6) Synchronize configuration. It will also setup persistent addresses
# smit hacmp
Extended Configuration → Extended Verification and Synchronization


# netstat –i

shows persistent IP addresses on the same interfaces as boot  addresses.

Cluster Resources Configuration

From one node, you will execute the following steps:
# smit hacmp
Extended Configuration →   Extended Resource Configuration →   Configure HACMP Service IP Labels/Addresses (srv address)
Define Application Servers
Define Resource Groups
Configure Resources for each Resource Group    
Customization
Synchronize Cluster Resources

Start HACMP on nodes

1) Configure HACMP Service IP Labels/Addresses

# smit hacmp
Extended Configuration →   Extended Resource Configuration →   HACMP Extended Resources Configuration →      Configure HACMP Service IP Labels/Addresses  →   Add a Service IP Label/Address
Select:                           Configurable on Multiple Nodes
Select a network name
Give interface service IP label or address (on the same subnet as “boot” address)
You shouldn’t  specify an Alternate HW Address to accompany IP Label/Address
because AIX 5L does “gratuitous ARP”  update.
Don’t  specify an Alternate HW Address for Ethernet Gigabit adapters

Repeat this step for ALL service IP labels.

2) Define Application Servers

# smit hacmp
Extended Configuration →   Extended Resource Configuration →   HACMP Extended Resources Configuration →     Configure HACMP Application Servers → Add an Application Server
Give:
- Server name (symbolic name for the resource)
- Start and Stop scripts full pathnames  (must exist on ALL NODES, in non-shared
filesystems).

Repeat this step for ALL Application Servers.

3) Define Resource Groups

# smit hacmp
Extended Configuration →   Extended Resource Configuration → HACMP Extended Resource Group Configuration →   Add a Resource Group
Select the Resource Group Management Policy: cascading, rotating, concurrent or custom
Give Resource Group Name,
Inter-Site Management Policy (leave default  ignore)
Give the list of Participating Node Names: for cascading the order defines the priority.

Repeat this step for all Resource Groups

4) Configure Resources for each Resource Group

# smit hacmp
Extended Configuration →   Extended Resource Configuration → HACMP Extended Resource Group Configuration → Change/Show Resources and Attributes for a Resource Group    
Select a Resource Group name
Define the resources belonging to the resource group (separated with space):
Inactive Takeover Applied   (true if you want first starting node to take all resources)
Cascading Without Fallback Enabled (true to decide when fallback will occur
recommended if HACMP  Cluster Services started in /etc/inittab)
Application Servers
Service IP Labels / Addresses  (Give ALL Service IP labels separated by space,
in case of several networks)
Volume Groups   give the name(s) separated by space
Use forced varyon of volume groups, if necessary (true for AIX mirrored VGs)
Filesystems (empty is All)   leave empty
Filesystems Recovery Method    leave sequential if nested recursive mount points
otherwise, parallel when many
Filesystems mounted before IP configured  (TRUE for NFS (cross)-mount)
Filesystems/Directories to Export     (must belong to Volume Groups listed before)
Filesystems/Directories to NFS mount (must belong to previous Export list)
Network for NFS mount

Repeat this step for all Resource Groups


Synchronize Cluster Resources (everytime you change the configuration)

# smit hacmp
Extended Configuration
Extended Verification and Synchronization
which allows to:
* Verify, Synchronize or Both                         [Both]       
Force synchronization if verification fails?        [No]         
* Verify changes only?     [No]          
* Logging               [Standard / Verbose]
In case of problem, select Verbose Logging and look to log files:
/var/hacmp/clverify/clverify.log  or /var/hacmp/clverify/…

Now you can start HACMP on all nodes (several nodes at the same time)

# smit clstart
* Start now, on system restart or both          now                    
Start Cluster Services on these nodes    [clusnod1]    you can specify several nodes             

Prerequisites

Before you start to set up the necessary file systems , carefully plan their distribution to available disks. Decide whether one or more volume groups (VGs) should be used.

Procedure

1. Enter the command smitty.
2. Select:
Physical & Logical Storage → Logical Volume Manager →
Volume Groups → Add a Volume Group
3. Enter a volume group name, for example, sapr3vg.
Press F4 to get a list of disks, which have not yet been assigned to a volume group.
4. With F7, mark the disks you want to be included in the volume group and press ENTER.
5. Press F10 to exit.
Repeat these steps for each volume group that has to be created.