Labels

Monitoring anti-affinity on XenServer

 
The article is not finished yet!

        In this article we will talk about monitoring "anti-affinity" groups of VMs running on XenServer hypervisor. Maybe no all of you know, what the anti-affinity, single point of failure means in virtualization. Let`s start little with the theory....

        Have you ever thought what would be single point of failure in your datacenter if you have multiple virtualization hosts? A single point of failure (SPOF) is a part of a system that, if it fails, will stop the entire system from working. The assessment of a potential SPOF involves identifying the critical components of a complex system that would provoke a total systems failure in case of malfunction. Highly reliable systems should not rely on any such individual component. So, what does it mean in ours virtualization in data center?

       Simple example: We have Windows domain with two Active Directory virtual machines (VMs). In hypervisor we have four hosts which can running lot of VMs. If both of Active Directory VMs will running on same host and this crashs, we lost Active Directory servers in one moment! For this scenarios VMware develop Anti-affinity groups feature read more .
Unfortunately XenServer doesn`t provide this feature. So how we can solve this issue in XenServer? You will find solution in this article.

         I have changed the approach to this problem. Instead I defined list of VMs which can`t run on same XenServer host, I will monitoring all Xen hosts if there running VMs from same cluster. For example servers from same cluster are fileserver01, fileserver02, fileserver03. Another cluster can by Graylog which consists from hosts: srvgraylog01,srvgraylog02. It is very common that servers with same functionality start with specific prefix like I mentioned above.

   Script in high level:
1. step 1
2. step 2
3. step 3
4. step 4


  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
######################################################################################################
#		Purpose:
#				Connect to XenServer and check if servers from any cluster run on same Xen host
#		Scriptname:
#				XenServer_antiaffinity_check.ps1
#		$ScriptVersion =  "1.2"
#		Prerequisites:
#				- "This script requires XenServer SDK"
#				- "Please download and install the version 7.3.0 of XenServer SDK"
#		Change history:
#			
#######################################################################################################

function Send-2Zabbix{
    param (
        [string]$ZabbixConfPol_key = "",
        [string]$ZabbixConfPol_value
    )
    $Zabbix_Config = "C:\Program Files\Zabbix Agent\zabbix_agentd.conf.scripts"
    $Zabbix_Sender = """C:\Program Files\Zabbix Agent\zabbix_sender.exe"""

    
    if (Test-path $Zabbix_Config) {
        $Zabbix_Config_Content = Get-Content $Zabbix_Config
        $Zabbix_Server_IP = $Zabbix_Config_Content | ? {$_ -like "ServerActive=*"}  | SELECT @{Name="Zabbix_IP"; Expression={$_ -replace "ServerActive=",""}} | Select -ExpandProperty "Zabbix_IP"
        $Zabbix_Client = "name_of_zabbix_client"

        if (!($Zabbix_Server_IP) -or !($Zabbix_Client ) ) {
            write-host "Zabbix configuration cannot be parsed"
            Exit
        }
    } else {
        write-host "Zabbix config not found"
        Exit
    }
    $result = ""
    #Send Output to Zabbix
    If ($ZabbixConfPol_key) {
        $cmd = "& $Zabbix_Sender -z '$Zabbix_Server_IP' -s '$Zabbix_Client' -k '$ZabbixConfPol_key' -o '$ZabbixConfPol_value'"
        Invoke-Expression $cmd
        Write-Host "sended to zabbix: $cmd"
    }
}

$ScriptPath = Split-Path $script:MyInvocation.MyCommand.Path
$start_time = Get-Date
Start-Transcript -Path "C:\Scripts\Logs\XenServer_affinity_log.txt"
# ======== For testing purpos use credentials from user input ==========================================
# $XenServer_credential = Get-Credential -Message "Credential are required for access to the XenServer"
# $Xenserver_UserName = $XenServer_credential.UserName
# $XenServer_Password = $XenServer_credential.GetNetworkCredential().Password
# ======================================================================================================
$Xenserver_UserName = "Xen_admin"
$file_with_pass = "C:\Scripts\Xen_admin_pass.txt"
$BSTR = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR((Get-Content $file_with_pass | ConvertTo-SecureString))
$XenServer_Password = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR)
[System.Runtime.InteropServices.Marshal]::ZeroFreeBSTR($BSTR)

$XenServerHost_list = @('xenHost1','xenHost2','xenHost3','xenHost4')  # list of pool masters
$cluster_exemption = @('mgmt','sup','sapserver')
# $VMs_exceptions = @('sapserver1','sapserver2') # <=== example: virtual machines which are allowed to run on same XenHost
$VMs_exceptions = @('some_hostname')
$result = @()
Import-Module 'XenServerPSModule'
if (Get-Module | Where-Object { $_.Name -eq "XenServerPSModule" }) {
    foreach ($XenServerHost in $XenServerHost_list) {
        Write-Host "$((Get-Date).ToString('T')) - Connecting to XenServer host: $XenServerHost"
        $session = Connect-XenServer -Server $XenServerHost -UserName $Xenserver_UserName -Password $XenServer_Password -NoWarnCertificates -SetDefaultSession -PassThru
        if ($session) {
            try {
                Write-Host "$((Get-Date).ToString('T')) - Loading list of MVs..."
                $VMs_temp = Get-XenVM | Where-Object{$_.is_a_template -eq $false -and $_.is_control_domain -eq $false -and $_.power_state -eq 'Running' `
                -and $_.name_label -notlike "*sap*" -and $_.name_label -notlike "Base*" -and $_.name_label -notlike "crm*"}
                $VMs = $VMs_temp | Select-Object @{n='VM_Name';e={$_.Name_label}},@{n='XenHost';e={((Get-XenVM -Name $_.Name_label | where-object{$_.is_a_template -eq $false}).resident_on | Get-XenHost).Name_label}} | Sort-Object -Property VM_name
                # VM list filtered from exemptions
                foreach ($VM_from_exemption in $cluster_exemption) {
                    $VMs = $VMs | Where-Object{$_.VM_Name -notlike "*$VM_from_exemption*"}
                }
                foreach ($VM in $VMs) {
                    # replace digits from VM name and set cluster name
                    $cluster = $VM.VM_name -replace '[^a-zA-Z-]',''
                    $bool_ignore_this_VM = $false
                    for ($i = 0; $i -lt $VMs.Count; $i++) {
                        write-host "Checking comparation VM: $($VM.VM_Name) with $($VMs[$i].VM_Name)"
                        if (($VMs[$i].VM_Name -replace '[^a-zA-Z-]','') -like "$cluster" -and $VMs[$i].VM_Name -notlike $VM.VM_Name) {
                            if ($VMs[$i].XenHost -eq $VM.XenHost) {
                                Write-Host "WARNING: comparation VM: $($VM.VM_Name) runs on $($VM.XenHost) - $($VMs[$i].VM_Name) runs on $($VMs[$i].XenHost)" -ForegroundColor Red
                                # check if current VM is in exemption list
                                foreach ($VMs_exception in $VMs_exceptions) {
                                    if ($VM.VM_Name -like $VMs_exception) {
                                        $bool_ignore_this_VM = $true
                                    }
                                }
                                if ($bool_ignore_this_VM -eq $false) {
                                    $result += "$cluster"
                                }
                            }
                            else {
                                Write-Host "INFO: comparation VM: $($VM.VM_Name) runs on $($VM.XenHost) - $($VMs[$i].VM_Name) runs on $($VMs[$i].XenHost)"
                            }
                        }
                    }
                    write-host "==="
                }
            }
            catch {
                Write-Host "$((Get-Date).ToString('T')) - Error: $($_.Exception.Message)" -ForegroundColor Red -BackgroundColor Yellow
                Break
            }
            finally {
                Disconnect-XenServer -Session $session
            }
        }
    }
    $result = $result | Select-Object -Unique
    if ($result.count -eq 0) {
        $zabbix_outup = "OK"
    }
    else {
        $zabbix_outup = $result -join ','
        write-host "Servers from cluster: $($result -join ',') run on same Xen hosts"
    }
    $end_time = get-Date
    $diff = New-TimeSpan -Start $start_time -End $end_time
    Write-Host "$((Get-Date).ToString('T')) - Script finished. Duration: $($diff)"
    # $exit = $host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
}
Send-2Zabbix -ZabbixConfPol_key "XenServer.antiaffinity" -ZabbixConfPol_value $zabbix_outup

Stop-Transcript
Labels:

Comments

Post a Comment

Popular Posts