Welcome! Log In Create A New Profile

Advanced

nginx as nonroot - setsockopt not permitted

Posted by orsolya.magos 
orsolya.magos
nginx as nonroot - setsockopt not permitted
September 14, 2018 10:00AM
Hi,

we use nginx which load-balances toward our snmptrapd. Everything is working
fine if we start nginx with root. We would like to change it so nginx
(workers) would start with nginx user. I couldn't make it work, do you have
any idea what additional thing can I set/check?

nginx -V
nginx version: nginx/1.12.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)
built with OpenSSL 1.0.2k-fips 26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx
--modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--http-client-body-temp-path=/var/lib/nginx/tmp/client_body
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid
--lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx
--with-file-aio --with-ipv6 --with-http_auth_request_module
--with-http_ssl_module --with-http_v2_module --with-http_realip_module
--with-http_addition_module --with-http_xslt_module=dynamic
--with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic
--with-http_sub_module --with-http_dav_module --with-http_flv_module
--with-http_mp4_module --with-http_gunzip_module
--with-http_gzip_static_module --with-http_random_index_module
--with-http_secure_link_module --with-http_degradation_module
--with-http_slice_module --with-http_stub_status_module
--with-http_perl_module=dynamic --with-mail=dynamic --with-mail_ssl_module
--with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module
--with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'
--with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-Wl,-E'

uname -a
Linux c-1 4.14.62-1.el7.centos.ncir.1.x86_64 #1 SMP Wed Aug 15 04:24:17 EEST
2018 x86_64 x86_64 x86_64 GNU/Linux


--------------------------------------------------------------------------------------------------

observation 0) with root user (master+workers)
everything works fine, snmptrapd gets the traps

--------------------------------------------------------------------------------------------------

observation 1)
idea: playing with setcap

config:
* with nginx user (master is root, workers are started with nginx user, so
in /etc/nginx/nginx.conf 'user nginx;' line is included)
root 2703077 0.0 0.0 59028 2280 ? Ss 11:34 0:00 nginx:
master process /usr/sbin/nginx
nginx 2703078 0.0 0.0 59476 4160 ? S 11:34 0:00 nginx:
worker process
nginx 2703079 0.0 0.0 59476 4840 ? S 11:34 0:00 nginx:
worker process
nginx 2703080 0.0 0.0 59476 4840 ? S 11:34 0:00 nginx:
worker process
.... etc.
* upstream port is 162, snmptrapd is listening there
* I've tried both capacities:
setcap cap_net_bind_service=+ep /sbin/nginx
setcap cap_net_admin+ep /sbin/nginx
* /etc/nginx/conf.d/stream/snmptrap.conf
upstream snmptrap_upstream {
#server x.y.z.226:162; #commented out for easier testing
#server x.y.z:227162; #commented out for easier testing
server x.y.z.228:162;
}
server {
listen z.y.z.225:162 udp;
proxy_pass snmptrap_upstream;
proxy_timeout 1s;
proxy_responses 0;
proxy_bind $remote_addr transparent;
error_log /var/log/nginx/snmptrap.log;
}
* also tried out switching off iptables



netstat -ulpn | grep 162
udp 0 0 x.y.z.228:162 0.0.0.0:*
2748327/snmptrapd
udp 0 0 x.y.z.225:162 0.0.0.0:*
2743096/nginx: mast


/var/log/nginx/snmptrap.log:
2018/09/12 11:55:04 [alert] 2739785#0: *23 setsockopt(IP_TRANSPARENT) failed
(1: Operation not permitted) while connecting to upstream, udp client:
x.y.z.225, server: x.y.z.225:162, upstream: "x.y.z.228:162", bytes from/to
client:5/0, bytes from/to upstream:0/0

/var/log/nginx/stream.log: error 500 is coming
2018-09-12T11:55:04+03:00 x.y.z.225 UDP 500 0 5 0.000 "0" "0" "0.000"

--------------------------------------------------------------------------------------------------
observation 2)
idea: trying an other upstream port (>1024), but still the same:

config:
* with nginx user (master is root, workers are started with nginx user, so
in /etc/nginx/nginx.conf 'user nginx;' line is included)
* upstream port is 4162
* /etc/nginx/conf.d/stream/snmptrap.conf
upstream snmptrap_upstream {
#server x.y.z.226:162; #commentedout for easier testing
#server x.y.z:227162; #commented out for easier testing
server x.y.z.228:4162;
}
server {
listen z.y.z.225:162 udp;
proxy_pass snmptrap_upstream;
proxy_timeout 1s;
proxy_responses 0;
proxy_bind $remote_addr transparent;
error_log /var/log/nginx/snmptrap.log;
}
* also tried out switching off iptables


netstat -ulpn | grep 162
udp 0 0 x.y.z.228:4162 0.0.0.0:*
2748327/snmptrapd
udp 0 0 x.y.z.225:162 0.0.0.0:*
2743096/nginx: mast


/var/log/nginx/snmptrap.log:
2018/09/12 11:08:03 [alert] 121472#0: *112642 setsockopt(IP_TRANSPARENT)
failed (1: Operation not permitted) while connecting to upstream, udp
client: x.y.z.225, server: x.y.z.225:162, upstream: "x.y.z.228:4162", bytes
from/to client:5/0, bytes from/to upstream:0/0

/var/log/nginx/stream.log: error 500 is coming
2018-09-12T11:08:03+03:00 x.y.z.225 UDP 500 0 5 0.000 "0" "0" "0.000"


Thanks in advance:
Orsi

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,281226,281226#msg-281226

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Maxim Dounin
Re: nginx as nonroot - setsockopt not permitted
September 14, 2018 02:00PM
Hello!

On Fri, Sep 14, 2018 at 03:52:03AM -0400, orsolya.magos wrote:

> we use nginx which load-balances toward our snmptrapd. Everything is working
> fine if we start nginx with root. We would like to change it so nginx
> (workers) would start with nginx user. I couldn't make it work, do you have
> any idea what additional thing can I set/check?
>
> nginx -V
> nginx version: nginx/1.12.2
> built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)

Update to nginx 1.13.8+, it should be able to use transparent
proxying on Linux without workers being run as root:

*) Feature: now nginx automatically preserves the CAP_NET_RAW capability
in worker processes when using the "transparent" parameter of the
"proxy_bind", "fastcgi_bind", "memcached_bind", "scgi_bind", and
"uwsgi_bind" directives.

Alternatively, consider not using "proxy_bind ... transparent".
See docs here for additional details:

http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_bind

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
orsolya.magos
Re: nginx as nonroot - setsockopt not permitted
September 14, 2018 03:20PM
Wow great, thanks Maxim for the super fast answer!
We are using epel version, still investigating the possibilities of version
change.
Br,
Orsi

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,281226,281230#msg-281230

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login