Welcome! Log In Create A New Profile

Advanced

req.body_param([<name>])

Posted by Simon Schabel 
Simon Schabel
req.body_param([<name>])
May 09, 2018 01:30PM
Hello,

We use the req.body_param([<name>]) setting to retrieve body parameter
from the incoming HTTP queries and place them into the logs.

Unfortunately this only works with HTTP POST requests. In our case we
need to extract the parameter from PUT requests as well.

Would it be an option to use req.body_param([<name>]) on any HTTP method
type instead of restricting it to HTTP POST?

Best
Simon

--
SEMKNOX GmbH
Simon Schabel
Webergasse 1, Haus B/1 · 01067 Dresden
simon.schabel@semknox.com · +49-351-32 123 102 · www.semknox.com
Jarno Huuskonen
Re: req.body_param([<name>])
May 11, 2018 12:50PM
Hi,

On Wed, May 09, Simon Schabel wrote:
> We use the req.body_param([<name>]) setting to retrieve body
> parameter from the incoming HTTP queries and place them into the
> logs.
>
> Unfortunately this only works with HTTP POST requests. In our case
> we need to extract the parameter from PUT requests as well.
>
> Would it be an option to use req.body_param([<name>]) on any HTTP
> method type instead of restricting it to HTTP POST?

Can you share your haproxy -vv and the logging config ?

I just tested with haproxy-ss-20180507 and this minimal test
seems to get req.body_param(log) to stick table on both POST/PUT requests
(tested w/curl -X PUT|POST):
....
frontend test
option http-buffer-request
bind ipv4@127.0.0.1:8080
http-request track-sc2 req.body_param(log),lower table test_be if METH_POST || METH_PUT

default_backend test_be

backend test_be
stick-table type string len 48 size 64 expire 240s store http_req_cnt
server wp2 some.ip.add.ress:80 id 2
....

curl -X PUT -d'log=user1' http://127.0.0.1:8080/
curl -X POST -d'log=user2' http://127.0.0.1:8080/

-Jarno

--
Jarno Huuskonen
Simon Schabel
Re: req.body_param([<name>])
May 14, 2018 01:30PM
Dear Jarno,

Thanks for your message.

Ok, my /haproxy -vv/ output is as follows:

HA-Proxy version 1.7.5-2~bpo8+1 2017/05/27
Copyright 2000-2017 Willy Tarreau <[email protected]>

Build options :
  TARGET  = linux2628
  CPU     = generic
  CC      = gcc
  CFLAGS  = -g -O2 -fPIE -fstack-protector-strong -Wformat
-Werror=format-security -D_FORTIFY_SOURCE=2
  OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1
USE_LUA=1 USE_PCRE=1 USE_NS=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.0.2k  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2l  25 May 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.35 2014-04-04
Running on PCRE version : 8.35 2014-04-04
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with Lua version : Lua 5.3.1
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Built with network namespace support

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
        [COMP] compression
        [TRACE] trace
        [SPOE] spoe



The setting for the logging was done in the /default /section as:

   log-format %Ci:%Cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %st\ %B\
%cc\ %cs\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\
%[capture.req.hdr(0)]\ %{+Q}r
        option log-separate-errors
        option log-health-checks

and in the /http /and /https /section the body parameter capturing is
activated as:

    # enable HTTP body logging
        option http-buffer-request
        declare capture request len 4
        http-request capture req.body_param(customerId) id 0

As my haproxy version differs from yours I'm unsure where I might made a
configuration error.

Thanks for your help,
best
Simon


Am 11.05.2018 um 12:43 schrieb Jarno Huuskonen:
> Hi,
>
> On Wed, May 09, Simon Schabel wrote:
>> We use the req.body_param([<name>]) setting to retrieve body
>> parameter from the incoming HTTP queries and place them into the
>> logs.
>>
>> Unfortunately this only works with HTTP POST requests. In our case
>> we need to extract the parameter from PUT requests as well.
>>
>> Would it be an option to use req.body_param([<name>]) on any HTTP
>> method type instead of restricting it to HTTP POST?
> Can you share your haproxy -vv and the logging config ?
>
> I just tested with haproxy-ss-20180507 and this minimal test
> seems to get req.body_param(log) to stick table on both POST/PUT requests
> (tested w/curl -X PUT|POST):
> ...
> frontend test
> option http-buffer-request
> bind ipv4@127.0.0.1:8080
> http-request track-sc2 req.body_param(log),lower table test_be if METH_POST || METH_PUT
>
> default_backend test_be
>
> backend test_be
> stick-table type string len 48 size 64 expire 240s store http_req_cnt
> server wp2 some.ip.add.ress:80 id 2
> ...
>
> curl -X PUT -d'log=user1' http://127.0.0.1:8080/
> curl -X POST -d'log=user2' http://127.0.0.1:8080/
>
> -Jarno
>

--
SEMKNOX GmbH
Simon Schabel
Webergasse 1, Haus B/1 · 01067 Dresden
simon.schabel@semknox.com · +49-351-32 123 102 · www.semknox.com
Jarno Huuskonen
Re: req.body_param([<name>])
May 14, 2018 02:50PM
Hi Simon,

On Mon, May 14, Simon Schabel wrote:
> HA-Proxy version 1.7.5-2~bpo8+1 2017/05/27
>
> The setting for the logging was done in the /default /section as:
>
>    log-format %Ci:%Cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %st\
> %B\ %cc\ %cs\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\
> %[capture.req.hdr(0)]\ %{+Q}r
>         option log-separate-errors
>         option log-health-checks
>
> and in the /http /and /https /section the body parameter capturing
> is activated as:
>
>     # enable HTTP body logging
>         option http-buffer-request
>         declare capture request len 4
>         http-request capture req.body_param(customerId) id 0
>
> As my haproxy version differs from yours I'm unsure where I might
> made a configuration error.

I tested with 1.8.8 and 1.7.5 and with both versions I managed to
log customerId (with simple curl -X PUT/POST).

Are the POST/PUT requests large, is it possible that the customerId doesn't
fit in haproxy buffer (default 16k (I think)) ?

Can you test with curl to see if customerId is logged then:
curl -v -X PUT -d'customerId=911' http://yourhost.yourdomain/yourpath

# bigfile is some random file much larger than 16k
and curl -v -X PUT [email protected] -d'customerId=912' http://yourhost.yourdomain/yourpath

-Jarno

--
Jarno Huuskonen
Sorry, only registered users may post in this forum.

Click here to login