Auto LUN balancing script v2.1

[Ernst]

Muggers,

Afgelopen weekend hebben wij meerdere storage paden toegevoed aan onze ESX config en ik zag dat mijn vorige script niet toereikend was.
Hieronder vind je de herschreven versie.

[Ernst]

Na het draaien van het script ziet je esxcfg-mpath -l er als volgt uit


Disk vmhba2:1:4 /dev/sdh (512000MB) has 4 paths and policy of Fixed
FC 16:0.1 50060b0000646c8a<->50060e8004f2e812 vmhba2:1:4 On active preferred
FC 16:0.1 50060b0000646c8a<->50060e8004f2e873 vmhba2:2:4 On
FC 19:0.1 50060b0000646062<->50060e8004f2e802 vmhba4:1:4 On
FC 19:0.1 50060b0000646062<->50060e8004f2e863 vmhba4:2:4 On


Disk vmhba2:1:5 /dev/sdi (512000MB) has 4 paths and policy of Fixed
FC 16:0.1 50060b0000646c8a<->50060e8004f2e812 vmhba2:1:5 On
FC 16:0.1 50060b0000646c8a<->50060e8004f2e873 vmhba2:2:5 On active preferred
FC 19:0.1 50060b0000646062<->50060e8004f2e802 vmhba4:1:5 On
FC 19:0.1 50060b0000646062<->50060e8004f2e863 vmhba4:2:5 On


Disk vmhba2:1:6 /dev/sdj (307200MB) has 4 paths and policy of Fixed
FC 16:0.1 50060b0000646c8a<->50060e8004f2e812 vmhba2:1:6 On
FC 16:0.1 50060b0000646c8a<->50060e8004f2e873 vmhba2:2:6 On
FC 19:0.1 50060b0000646062<->50060e8004f2e802 vmhba4:1:6 On active preferred
FC 19:0.1 50060b0000646062<->50060e8004f2e863 vmhba4:2:6 On


Disk vmhba2:1:7 /dev/sdk (307200MB) has 4 paths and policy of Fixed
FC 16:0.1 50060b0000646c8a<->50060e8004f2e812 vmhba2:1:7 On
FC 16:0.1 50060b0000646c8a<->50060e8004f2e873 vmhba2:2:7 On
FC 19:0.1 50060b0000646062<->50060e8004f2e802 vmhba4:1:7 On
FC 19:0.1 50060b0000646062<->50060e8004f2e863 vmhba4:2:7 On active preferred


Disk vmhba2:1:8 /dev/sdl (512000MB) has 4 paths and policy of Fixed
FC 16:0.1 50060b0000646c8a<->50060e8004f2e812 vmhba2:1:8 On active preferred
FC 16:0.1 50060b0000646c8a<->50060e8004f2e873 vmhba2:2:8 On
FC 19:0.1 50060b0000646062<->50060e8004f2e802 vmhba4:1:8 On
FC 19:0.1 50060b0000646062<->50060e8004f2e863 vmhba4:2:8 On


Disk vmhba2:1:9 /dev/sdm (512000MB) has 4 paths and policy of Fixed
FC 16:0.1 50060b0000646c8a<->50060e8004f2e812 vmhba2:1:9 On
FC 16:0.1 50060b0000646c8a<->50060e8004f2e873 vmhba2:2:9 On active preferred
FC 19:0.1 50060b0000646062<->50060e8004f2e802 vmhba4:1:9 On
FC 19:0.1 50060b0000646062<->50060e8004f2e863 vmhba4:2:9 On

[duncanepping]

Ernst is het goed als ik op mijn blog publiceer? uiteraard met jouw naam er bij?
_________________
Nu beschikbaar: vSphere 4.0 Quick Start Guide!

[Ernst]

Tuurlijk geen probleem. Lang leven Opensource zeg ik.

Wel ff linkje hier post he

[esloof]

Nou Duncan,

Je bent me net voor, ik had Ernst vanmiddag nog aan de lijn over het posten van dit wereld script.

Groeten,

Eric Sloof

[Ernst]

Ok jullie mogen het allebei posten.

Wordt er helemaal van.

[duncanepping]

http://www.yellow-bricks.com/2008/04/01/load-balancing-activeactive-sans/
_________________
Nu beschikbaar: vSphere 4.0 Quick Start Guide!

[Pauli]

I tried the script but I do not have consistent results: reason of this is that if I run the cmd "esxcfg-mpath -l" I got also two SCSI3-controllers - and the order of these devices is not equal on the esx hosts .
- config 6 hosts BL680 with 4 quad Xeon
- 2 EVA8000 with 8 paths to each LUN

Can the script be adapted to make it consistent ?

[Pauli]

I found it : just another grep to take only the hba into account and now it looks more consistent: see at the end of the "for LUN" statement


for PATHS in 2 4 6 8
do
STPATHS=${PATHS}
COUNTER="1"

for LUN in $(esxcfg-mpath -l | grep "has ${STPATHS} paths" | awk '{print $2}' | grep "hba")
do
esxcfg-mpath --lun=${LUN} --path=$(esxcfg-mpath -q --lun=${LUN} | grep FC | awk '{print $4}' | awk '{print NR "S\t " $0}' | grep ${COUNTER}S | awk '{print $2}') --preferred
COUNT=`expr ${COUNTER} + 1`
COUNTER=${COUNT}
if [[ ${COUNTER} -gt ${STPATHS} ]]
then
COUNTER="1"
fi
done
done

[Ernst]

I found out that every LUN is presented over the same path for each ESX server. If a mod x calculation is used if you have numbers in your hostname its relative easy to make sure all paths are used equally and your storage FA is not overloaded via 1 path.

Before:
host1-lun1-path1
host2-lun1-path1
host3-lun1-path1
host4-lun1-path1

After:
host1-lun1-path1
host2-lun1-path2
host3-lun1-path3
host4-lun1-path4

etc.

e.g. take the following hostsnames as an example

esx010
esx011
esx012
esx013


for PATHS in 2 4 6 8
do
echo PATHS: $PATHS
STPATHS=${PATHS}
COUNTER=`hostname|sed -e 's/esx//'|sed -e 's/^0//'`
let COUNTER=$COUNTER%$PATHS+1

for LUN in $(esxcfg-mpath -l | grep "has ${STPATHS} paths" | awk '{print $2}')
do
esxcfg-mpath --lun=${LUN} --path=$(esxcfg-mpath -q --lun=${LUN} | grep FC | awk '{print $4}' | awk '{print NR "S\t " $0}' | grep ${COUNTER}S | awk '{print $2}') --preferred
COUNT=`expr ${COUNTER} + 1`
COUNTER=${COUNT}
if [[ ${COUNTER} -gt ${STPATHS} ]]
then
COUNTER="1"
fi
done
done

[Pauli]

I thought the performance of the SAN for a certein LUN is higher when a LUN is always access via the same path: (with HP EVA8000). Reason is that EVA implements a concept of LUN-ownership. This means that a LUN is always accessed by the same controller. So when you spread the paths of a certain LUN this will result in using both controllers and that when the non-owning controller receives the request it will pass this to the other controller resulting in suboptimal response.
EVA8000 has two controllers, so it can be better in terms of Fiber optimization that the paths of a certain LUN is spread over the 4 paths of the owning controller.
EVA8000: two controllers with each has 4 paths.
Ie. LUN 1 : path1 on host1,path2 on host2,path3 on host3,path4 on host4
path1 on host5, path2 on host6
LUN 2 : path5 on host1,path6 on host2,path7 on host3,path8 on host4
path5 on host5, path6 on host6

One other thing to take into account is the LOAD the EVA itself places on the fiber LUNs of the controller for Continous Access: with the current firmware it will uses the even port numbers for the replication traffic. So maybe it is better to use the odd port numbers more than the even port numbers.
Anyway, best is to measure via EVAPERF how the load is on the fiberports of the controllers.

[Ernst]

I must totally agree but for the XP1200 & XP24000 this is not exactly the case. HP engineers also thought that this was the case that the STP's of the XP do not handle intercommunication well and that this would slow the whole box down. We tried to work to a sollution with HP and we see now that balancing the LUNs over different storage paths helps the STP's act more relaxt when experiencing heavy load.

Next week we are going to do some test with HP to see which give the best performance.

Ernst

[Ernst]

OK HP came up with the following:

On the XP1200 the inter SP communication is not optimal there for presenting a LUN over different storagepaths on different ESX servers is not optimal.

This is different for every storage array and should implemented when the storage array vendor advices it.

So my advice is (if your not sure) use the original /1st one

Ernst

[sontom115]

Welcom
I am newbie, good day

pret auto

[DukeBox]