Error with haproxy using balance roundrobin to connect to long-polling CometD server?

Hi,

I have two CometD servers, They use *long-polling* to push messages to
users.
cometd1: 10.10.69.103
cometd2: 10.11.157.139
user A can use "http://10.10.69.103:80/?user=A"; to connect to cometd1

Now I have haproxy on 10.10.69.102, haproxy.cfg like this:

global
log 127.0.0.1 local0
maxconn 4096
chroot /opt/apps/haproxy
uid 99
gid 99
daemon
nbproc 1
pidfile /opt/apps/haproxy/logs/haproxy.pid

defaults
log 127.0.0.1 local3
mode http
option httplog
option httpclose
option dontlognull
option forwardfor
option redispatch
retries 2
stats uri /haproxy-stats
contimeout 5000
clitimeout 50000
srvtimeout 50000


backend cometd
mode http
timeout connect 5s
timeout server 5m
*balance source*
server cometd1 10.10.69.103:80 check weight 12 maxconn 100000
server cometd2 10.11.157.139:80 check weight 13 maxconn 200000

frontend http_proxy
bind *:80
mode http
option forwardfor
option http-server-close
option http-pretend-keepalive
default_backend cometd

It works well.

When I open url: http://10.10.69.102?user=lihaibo6, the haproxy log like
this:

Mar 8 11:21:58 localhost.localdomain haproxy[9148]:
10.1.80.136:1972[08/Mar/2012:11:21:58.042] http_proxy cometd/
*cometd1* 0/0/0/2/2 200 1208 - - ---- 0/0/0/0/0 0/0 "GET /?user=lihaibo6
HTTP/1.1"
Mar 8 11:21:58 localhost.localdomain haproxy[9148]:
10.1.80.136:1979[08/Mar/2012:11:21:58.262] http_proxy cometd/
*cometd1* 0/0/0/1/2 400 1627 - - ---- 0/0/0/0/0 0/0 "GET /cometd HTTP/1.1"
Mar 8 11:21:59 localhost.localdomain haproxy[9148]:
10.1.80.136:1983[08/Mar/2012:11:21:59.335] http_proxy cometd/
*cometd1* 0/0/0/2/2 200 355 - - ---- 0/0/0/0/0 0/0 "POST /cometd/handshake
HTTP/1.1"
Mar 8 11:21:59 localhost.localdomain haproxy[9148]:
10.1.80.136:1984[08/Mar/2012:11:21:59.367] http_proxy cometd/
*cometd1* 0/0/0/2/2 200 288 - - ---- 0/0/0/0/0 0/0 "POST /cometd/ HTTP/1.1"
Mar 8 11:21:59 localhost.localdomain haproxy[9148]:
10.1.80.136:1985[08/Mar/2012:11:21:59.401] http_proxy cometd/
*cometd1* 0/0/0/2/2 200 271 - - ---- 0/0/0/0/0 0/0 "POST /cometd/connect
HTTP/1.1"

It seems take *5 steps(comet connect, comet handshake...)* to establish a
long-polling cometd connection.

But, If I use balance roundrobin, not balance source, It can not work, and
I get haproxy log like this:

Mar 8 11:19:44 localhost.localdomain haproxy[9124]:
10.1.80.136:1791[08/Mar/2012:11:19:44.345] http_proxy cometd/
*cometd2* 0/0/0/3/3 200 1111 - - ---- 0/0/0/0/0 0/0 "GET /?user=lihaibo6
HTTP/1.1"
Mar 8 11:19:44 localhost.localdomain haproxy[9124]:
10.1.80.136:1792[08/Mar/2012:11:19:44.509] http_proxy cometd/
*cometd2* 0/0/0/2/2 400 1627 - - ---- 0/0/0/0/0 0/0 "GET /cometd HTTP/1.1"
Mar 8 11:19:45 localhost.localdomain haproxy[9124]:
10.1.80.136:1794[08/Mar/2012:11:19:45.535] http_proxy cometd/
*cometd1* 0/0/0/2/2 200 356 - - ---- 0/0/0/0/0 0/0 "POST /cometd/handshake
HTTP/1.1"
Mar 8 11:19:45 localhost.localdomain haproxy[9124]:
10.1.80.136:1795[08/Mar/2012:11:19:45.566] http_proxy cometd/
*cometd2* 0/0/0/2/2 200 326 - - ---- 0/0/0/0/0 0/0 "POST /cometd/ HTTP/1.1"
Mar 8 11:19:45 localhost.localdomain haproxy[9124]:
10.1.80.136:1796[08/Mar/2012:11:19:45.598] http_proxy cometd/cometd1
0/0/0/1/1 200 271 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:45 localhost.localdomain haproxy[9124]:
10.1.80.136:1797[08/Mar/2012:11:19:45.629] http_proxy cometd/cometd2
0/0/0/2/2 200 290 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:45 localhost.localdomain haproxy[9124]:
10.1.80.136:1798[08/Mar/2012:11:19:45.675] http_proxy cometd/cometd1
0/0/0/1/1 400 1627 - -
---- 0/0/0/0/0 0/0 "GET /cometd HTTP/1.1"
Mar 8 11:19:46 localhost.localdomain haproxy[9124]:
10.1.80.136:1801[08/Mar/2012:11:19:46.775] http_proxy cometd/cometd2
0/0/0/2/2 200 355 - -
---- 0/0/0/0/0 0/0 "POST /cometd/handshake HTTP/1.1"
Mar 8 11:19:46 localhost.localdomain haproxy[9124]:
10.1.80.136:1802[08/Mar/2012:11:19:46.790] http_proxy cometd/cometd1
0/0/0/2/2 200 327 - -
---- 0/0/0/0/0 0/0 "POST /cometd/ HTTP/1.1"
Mar 8 11:19:46 localhost.localdomain haproxy[9124]:
10.1.80.136:1803[08/Mar/2012:11:19:46.801] http_proxy cometd/cometd2
0/0/0/2/2 200 272 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:46 localhost.localdomain haproxy[9124]:
10.1.80.136:1804[08/Mar/2012:11:19:46.815] http_proxy cometd/cometd1
0/0/0/1/1 200 291 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:46 localhost.localdomain haproxy[9124]:
10.1.80.136:1805[08/Mar/2012:11:19:46.825] http_proxy cometd/cometd2
0/0/0/1/2 400 1627 - -
---- 0/0/0/0/0 0/0 "GET /cometd HTTP/1.1"
Mar 8 11:19:47 localhost.localdomain haproxy[9124]:
10.1.80.136:1806[08/Mar/2012:11:19:47.850] http_proxy cometd/cometd1
0/0/0/1/1 200 357 - -
---- 0/0/0/0/0 0/0 "POST /cometd/handshake HTTP/1.1"
Mar 8 11:19:47 localhost.localdomain haproxy[9124]:
10.1.80.136:1807[08/Mar/2012:11:19:47.881] http_proxy cometd/cometd2
0/0/0/2/2 200 328 - -
---- 0/0/0/0/0 0/0 "POST /cometd/ HTTP/1.1"
Mar 8 11:19:47 localhost.localdomain haproxy[9124]:
10.1.80.136:1808[08/Mar/2012:11:19:47.911] http_proxy cometd/cometd1
0/0/0/1/1 200 272 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:47 localhost.localdomain haproxy[9124]:
10.1.80.136:1809[08/Mar/2012:11:19:47.943] http_proxy cometd/cometd2
0/0/0/2/2 200 291 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:47 localhost.localdomain haproxy[9124]:
10.1.80.136:1810[08/Mar/2012:11:19:47.981] http_proxy cometd/cometd1
0/0/0/0/1 400 1627 - -
---- 0/0/0/0/0 0/0 "GET /cometd HTTP/1.1"
Mar 8 11:19:49 localhost.localdomain haproxy[9124]:
10.1.80.136:1813[08/Mar/2012:11:19:49.021] http_proxy cometd/cometd2
0/0/0/2/2 200 358 - -
---- 0/0/0/0/0 0/0 "POST /cometd/handshake HTTP/1.1"
Mar 8 11:19:49 localhost.localdomain haproxy[9124]:
10.1.80.136:1814[08/Mar/2012:11:19:49.053] http_proxy cometd/cometd1
0/0/0/1/1 200 328 - -
---- 0/0/0/0/0 0/0 "POST /cometd/ HTTP/1.1"
Mar 8 11:19:49 localhost.localdomain haproxy[9124]:
10.1.80.136:1815[08/Mar/2012:11:19:49.094] http_proxy cometd/cometd2
0/0/0/2/2 200 272 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:49 localhost.localdomain haproxy[9124]:
10.1.80.136:1816[08/Mar/2012:11:19:49.125] http_proxy cometd/cometd1
0/0/0/1/1 200 291 - -
---- 0/0/0/0/0 0/0 "POST /cometd/connect HTTP/1.1"
Mar 8 11:19:49 localhost.localdomain haproxy[9124]:
10.1.80.136:1817[08/Mar/2012:11:19:49.255] http_proxy cometd/cometd2
0/0/0/2/2 400 1627 - -
---- 0/0/0/0/0 0/0 "GET /cometd HTTP/1.1"

It keep attempting to connect to CometD servers, but fails all the time,
because "balance roundrobin" can not make sure the "*5 steps(comet connect,
comet handshake...)* " on the same server, right?
Is this a bug?



--
Best Regard!

Yes, that is a bug in your configuration. You need to tell haproxy the
connections needs to go to that same server if that is what the servers
need. When I require session affinity, personally I prefer to use cookies
to make that happen but they might not work for some situations.



Add/adjust something like the following for your backend section to use
cookies:

cookie COMETD insert

server cometd1 10.10.69.103:80 check cookie c1 weight 12 maxconn
100000

server cometd2 10.11.157.139:80 check cookie c2 weight 13 maxconn
200000









From: 李海波 [mailto:[email protected]]
Sent: Wednesday, March 07, 2012 11:39 PM
To: haproxy@formilux.org
Subject: Error with haproxy using balance roundrobin to connect to
long-polling CometD server?







It keep attempting to connect to CometD servers, but fails all the time,
because "balance roundrobin" can not make sure the "5 steps(comet connect,
comet handshake...) " on the same server, right?

Is this a bug?

Hi

"balance source" is more affinity than persistence, which means that
you might be redirected to an other server sometime.
To be very efficient, source balancing needs a very large scope of source IPs.
You can try mixing balance source with "hash-type consistent" to try
to avoid your client to switch from a server to an other one.

Since you're doing long polling I would recommend balance leastconn to
balance better te load.
If you really need persistence, then you'll have to use either "cookie
insert" or "cookie prefix".

cheers