I was using the latest version of HAProxy 1.3 and was load balancing backend MySQL servers while also checking their ports, so if one server went down it would be taken out of the load balancing pool. However, since the port checks in HAProxy happen at the TCP level, the MySQL instance which was being hit by the port checks wasn't happy, because it wasn't a proper MySQL connection. As a result, after a number of some checks, MySQL refused to allow clients to connect, with a message like this:
OperationalError: (1129, "Host 'myhost' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'")
Solution: upgrade HAProxy to the newly released version 1.4 (at the date of this writing, the exact version is 1.4.0). Documentation is here.
For MySQL specific checks, you can specify 'option mysql-check' in a backend or 'listen' section of the configuration file. For example, I have something similar to this in my HAProxy configuration file:
listen mysql 0.0.0.0:3306
server mysql1 10.0.1.1:3306 check port 3306
server mysql2 10.0.1.2:3306 check port 3306 backup
Saturday, February 27, 2010
If you run your infrastructure in AWS, then you can use CloudWatch Logs and AWS ElasticSearch + Kibana for log aggregation/searching/visuali...
Here's a good interview question for a tester: how do you define performance/load/stress testing? Many times people use these terms inte...
Gatling is a modern load testing tool written in Scala. As part of the Jenkins setup I am in charge of , I wanted to run load tests using Ga...
I first saw nsupdate mentioned on the devops-toolchain mailing list as a tool for dynamically updating DNS zone files from the command line....