Orchestrator datacenter detection

Posted in: MySQL, Open Source, Technical Track

Orchestrator is datacenter-aware, meaning that it takes nodes location into consideration when performing a topology change. Let’s explore the options to make Orchestrator automatically detect this configuration.

 

Using MySQL server

One option is to use the monitored database server to retrieve the datacenter. This is enabled by the DetectDataCenterQuery Orchestrator parameter. A simple approach (which is also suggested in the documentation) is to use string functions to extract the datacenter name or equivalent from the hostname:

"DetectDataCenterQuery": "select substring_index(substring_index(@@hostname, '.',2), '.', -1) as dc",

For a server with hostname mysql01.us-east.myorg.com, the above query would return us-east.

In case extracting the location from the hostname is not possible, we could create an auxiliary table to store hosts and locations:


mysql> select * from host_dc;
+------+----------+---------+
| id | hostname | dc |
+------+----------+---------+
| 1 | mysql01 | us-east |
| 2 | mysql02 | us-west |
| 3 | mysql03 | eu-west |
+------+----------+---------+

and using a query like the following should be enough: select dc from host_dc where hostname = @@hostname. Of course, the disadvantage of this approach is evident: you need to keep the auxiliary table up to date.

Using regular expressions

This method is a variant of the first one, but instead of using MySQL functions, we use regular expressions. The Orchestrator parameter behind it is DataCenterPattern. Now, one of the main reasons I thought of creating this blog post was to share how this parameter works, as it was not straightforward for me. Let’s take a look at the default value:

"DataCenterPattern": "[.]([^.]+)[.][^.]+[.]mydomain[.]com",

Searching for the parameter in the code, we can see how this expression is being used (instance_dao.go):

if config.Config.DataCenterPattern != "" {
		if pattern, err := regexp.Compile(config.Config.DataCenterPattern); err == nil {
			match := pattern.FindStringSubmatch(instance.Key.Hostname)
			if len(match) != 0 {
				instance.DataCenter = match[1]
			}
		}
		// This can be overriden by later invocation of DetectDataCenterQuery
	}

Reading GO documentation on function FindStringSubmatch, we can understand better how parentheses play in this expression; the function will return an array including the substring matching the regular expression as expected, but also any substrings matching the portions of the regex in parentheses. Orchestrator code above shows that position “1” of the returned array will be used as the datacenter identifier, so we need to make the first portion of the regex in parentheses to match this.

Perhaps an example will help clarify the paragraph above. If we have hostnames with the format mysqlNN.dcX and we want to use dcX as the datacenter identifier, we need to create an expression that matches the whole hostname first, and then include the regex portion we want for the datacenter in parentheses. Something like this should work: mysql.*[.](.*). Using the code test in golang.org, we can see what’s returned:

package main
import (
"fmt"
"regexp"
)
func main() {
   re := regexp.MustCompile("mysql.*[.](.*)")
   fmt.Printf("%q\n", re.FindStringSubmatch("mysql01.dc1"))
}
------------------------
["mysql01.dc1" "dc1"]

Conclusion

There are two ways to get Orchestrator to automatically detect the datacenter. The first one relies on a MySQL connection against the monitored host and can be based on server variables or an auxiliary table, although the latter requires datacenter metadata to be kept up to date.

The second approach relies on applying a regular expression to the hostname, but with the particularity that a portion of the expression that would correspond to the datacenter identifier needs to be in parentheses.

email
Want to talk with an expert? Schedule a call with our team to get the conversation started.

Internal Principal Consultant

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *