Hi Willy,
Thanks so much for your responses! We're using version 1.4.19. I'll work
on getting a network capture for you -- what tcpdump params would help the
most? I usually use "-vv -A -w <file>". Probably the most helpful thing
would be to dump port 81, which is the unencrypted traffic coming it
HAProxy from stunnel.
Thanks again,
rusty
On 3/27/12 2:21 PM, "Willy Tarreau" <
[email protected]> wrote:
>Hi Rusty,
>
>On Mon, Mar 26, 2012 at 02:07:40PM -0500, Rusty Geldmacher wrote:
>> Hi all,
>>
>> Was hoping someone could help shed some light on an issue we're
>>observing in
>> our setup. When we have "option http-server-close" set, then file
>>uploads via
>> multipart form data will not work. By "not work" I mean that the HTTP
>> content-length header will be reported as 0, which in turn causes the
>>final
>> destination to get seriously confused by the request and then give up. I
>> realize this sounds strange, as content-length is reported by the
>>client but
>> this is the behavior I'm witnessing. When I have http-server-close on,
>>the
>> content-length is 0, and when I remove the option the content-length is
>>as
>> expected ? with no changes on the client end.
>
>That's strange indeed. What version is this, so that we can check the
>changelog if any known bug might have had that side effect ?
>
>> Additionally, if we set "option httpclose" then everything will work.
>
>I'm not surprized, "option httpclose" only adjusts the Connection header
>and completely ignores framing. It's what we call the tunnel mode : both
>sides are free to exchange what they want past the headers.
>
>> However, I'm under the impression that using httpclose isn't ideal since
>> clients then wouldn't be able to send multiple HTTP requests across the
>>same
>> connection.
>
>You're right.
>
>> Since this is a mobile app, having that functionality would
>> enhance performance.
>
>For sure !
>
>> Setting http-server-close seems to be ideal because we
>> can correctly load balance multiple requests in the same session.
>
>Yes and it was designed exactly for this.
>
>> So, I'm wondering:
>>
>> * Does haproxy do anything to validate/calculate the content-length
>>header?
>
>Yes, it parses the Content-length header or alternatively considers the
>Transfer-Encoding if it's present and announced as "chunked".
>
>> * How would the presence of http-server-close affect the request's
>>content-length header?
>
>It only checks the header and should not mangle it. Well, there is only
>one
>case it changes it, it's when multiple similar content-length headers are
>sent, then it reduces them to only one.
>
>> At a high level, our request flow looks like this:
>>
>> stunnel -> haproxy -> apache -> passenger
>>
>> Where stunnel and haproxy are on the same machine, and apache/passenger
>>are
>> on three or four additional machines.
>>
>> Here's the relevant snippet from haproxy.cfg:
>>
>> defaults
>> mode http
>> log global
>> option dontlognull
>> option redispatch
>> retries 3
>> timeout http-request 10s
>> timeout queue 1m
>> timeout connect 10s
>> timeout client 1m
>> timeout server 1m
>> timeout http-keep-alive 500
>> timeout check 10s
>> maxconn 3000
>>
>> # Stunnel listens on 443 and forwards connections off to haproxy on 81
>> frontend https *:81
>> option httplog
>> # Enabling http-server-close will cause file uploads to fail!
>> # option http-server-close
>> option forwardfor except 127.0.0.0/8
>>
>> reqadd X-Forwarded-Proto:\ https
>>
>> # set some ACLS?
>> # choose a backend?
>> # Nothing else really special
>
>Nothing fancy here !
>
>> And here is a session log for each case, starting with the normal
>>operation
>> (no http-server-close set, or setting httpclose):
>>
>> 00000020:https.accept(0006)=000a from [10.0.1.30:43419]
>> 00000020:https.clireq[000a:ffff]: POST /URL-GOES-HERE HTTP/1.1
>> 00000020:https.clihdr[000a:ffff]: Host: dev.sermo.com
>> 00000020:https.clihdr[000a:ffff]: Content-Type: multipart/form-data;
>>charset=utf-8; boundary=0xKhTmLbOuNdArY
>> 00000020:https.clihdr[000a:ffff]: User-Agent: iPhone Simulator; iPhone
>>OS 4.3.2; en_US
>> 00000020:https.clihdr[000a:ffff]: Accept-Encoding: gzip
>> 00000020:https.clihdr[000a:ffff]: Content-Length: 62528
>> 00000020:https.clihdr[000a:ffff]: Connection: keep-alive
>>
>> And here is with nothing else changed except for the addition of the
>> http-server-close option:
>>
>> 00000007:https.accept(0006)=000a from [10.0.1.30:43688]
>> 00000007:https.clireq[000a:ffff]: POST /URL-GOES-HERE HTTP/1.1
>> 00000007:https.clihdr[000a:ffff]: Host: dev.sermo.com
>> 00000007:https.clihdr[000a:ffff]: Content-Type: multipart/form-data;
>>charset=utf-8; boundary=0xKhTmLbOuNdArY
>> 00000007:https.clihdr[000a:ffff]: User-Agent: iPhone Simulator; iPhone
>>OS 4.3.2; en_US
>> 00000007:https.clihdr[000a:ffff]: Accept-Encoding: gzip
>> 00000007:https.clihdr[000a:ffff]: Content-Length: 0
>> 00000007:https.clihdr[000a:ffff]: Connection: keep-alive
>
>Wow this is amazing !
>I've never seen that and don't understand how that may happen. I'm relly
>interested in knowing your version to try to reproduce.
>
>> We can get the system to work if instead of setting http-server-close,
>>we set
>> httpclose but like I explained above this doesn't seem to be ideal.
>
>I agree this is not a good solution, we need to fix this.
>
>If you could get a network capture for each side too, that would help
>a lot. Don't post it to the list if there are sensible information there.
>
>Regards,
>Willy
>