Welcome! Log In Create A New Profile

Advanced

Performance of using lua calls for map manipulation on every request

Posted by Sachin Shetty 
Hi,

We are doing about 10K requests/minute on a single haproxy server, we have
enough CPUs and memory. Right now each requests looks up a map for backend
info. It works well.

Now we need to build some expire logic around the map. Like ignore some
entries in the map entries after some time. I could do this in lua, but it
woud mean that every request would make a lua call to look up a map value
and make a decision.

My lua method looks like this:

function get_proxy_from_map(txn)
local host = txn.http:req_get_headers()["host"][0]
local value = proxy_map_v2:lookup(host)
if value then
local values = split(value, ",")
local proxy = values[1]
local time = values[2]
if os.time() > tonumber(time) then
core.Alert("Expired: returning nil: " .. host)
return
else
return proxy
end
end
return
end


Any suggestions on how this would impact performance, our tests looks ok.

Thanks
Sachin
Hi,

The Lua overhead is very low. On my laptop I reach easyly 18 000 HTTP
requests by seconds with basic Lua processing. I guess that your code
will not have significant impact on perfs.

Note that the function:

> txn.http:req_get_headers()["host"][0]

Is consume more CPU than

txn.f[‘req.fhdr’](‘host’)

or

txn.sf[‘req.fhdr’](‘host’)

Other point: I’m not sure that the split() function exists.

Thierry


> On 27 Jul 2018, at 14:38, Sachin Shetty <[email protected]> wrote:
>
> Hi,
>
> We are doing about 10K requests/minute on a single haproxy server, we have enough CPUs and memory. Right now each requests looks up a map for backend info. It works well.
>
> Now we need to build some expire logic around the map. Like ignore some entries in the map entries after some time. I could do this in lua, but it woud mean that every request would make a lua call to look up a map value and make a decision.
>
> My lua method looks like this:
>
> function get_proxy_from_map(txn)
> local host = txn.http:req_get_headers()["host"][0]
> local value = proxy_map_v2:lookup(host)
> if value then
> local values = split(value, ",")
> local proxy = values[1]
> local time = values[2]
> if os.time() > tonumber(time) then
> core.Alert("Expired: returning nil: " .. host)
> return
> else
> return proxy
> end
> end
> return
> end
>
>
> Any suggestions on how this would impact performance, our tests looks ok.
>
> Thanks
> Sachin
Thankyou Thierry for your reply. I will change to txn.f[‘req.fhdr’].

On Wed, Aug 1, 2018 at 2:31 PM, Thierry Fournier <
[email protected]> wrote:

> Hi,
>
> The Lua overhead is very low. On my laptop I reach easyly 18 000 HTTP
> requests by seconds with basic Lua processing. I guess that your code
> will not have significant impact on perfs.
>
> Note that the function:
>
> > txn.http:req_get_headers()["host"][0]
>
> Is consume more CPU than
>
> txn.f[‘req.fhdr’](‘host’)
>
> or
>
> txn.sf[‘req.fhdr’](‘host’)
>
> Other point: I’m not sure that the split() function exists.
>
> Thierry
>
>
> > On 27 Jul 2018, at 14:38, Sachin Shetty <[email protected]> wrote:
> >
> > Hi,
> >
> > We are doing about 10K requests/minute on a single haproxy server, we
> have enough CPUs and memory. Right now each requests looks up a map for
> backend info. It works well.
> >
> > Now we need to build some expire logic around the map. Like ignore some
> entries in the map entries after some time. I could do this in lua, but it
> woud mean that every request would make a lua call to look up a map value
> and make a decision.
> >
> > My lua method looks like this:
> >
> > function get_proxy_from_map(txn)
> > local host = txn.http:req_get_headers()["host"][0]
> > local value = proxy_map_v2:lookup(host)
> > if value then
> > local values = split(value, ",")
> > local proxy = values[1]
> > local time = values[2]
> > if os.time() > tonumber(time) then
> > core.Alert("Expired: returning nil: " .. host)
> > return
> > else
> > return proxy
> > end
> > end
> > return
> > end
> >
> >
> > Any suggestions on how this would impact performance, our tests looks
> ok.
> >
> > Thanks
> > Sachin
>
>
Sorry, only registered users may post in this forum.

Click here to login