Welcome! Log In Create A New Profile

Advanced

1.8 resolvers - start vs. run

Posted by Jim Freeman 
Jim Freeman
1.8 resolvers - start vs. run
December 29, 2017 07:10PM
I'm a bit befuddled by the different nameserver config 'twixt these 2 modes?
[ Methinks I grok the need for an internal non-libc/libresolv resolver ]

Why isn't the the /etc/resolv.conf start-time config used (or at least
available) as a default run-time config (chroot notwithstanding)?
Under what circumstances do nameservers/settings need to be different in
start vs. run modes?

I'd expect that for most installations, the run-time config could/should be
the same as the start-time config ? Having to create a run-time config
that will just be the same as the start-time gets in the way of automating
of config across different environments ...

Or am I just not reading the docs right ?
Lukas Tribus
Re: 1.8 resolvers - start vs. run
December 29, 2017 09:10PM
Hello,


On Fri, Dec 29, 2017 at 7:00 PM, Jim Freeman <[email protected]> wrote:
> I'm a bit befuddled by the different nameserver config 'twixt these 2 modes?
> [ Methinks I grok the need for an internal non-libc/libresolv resolver ]
>
> Why isn't the the /etc/resolv.conf start-time config used (or at least
> available) as a default run-time config (chroot notwithstanding)?
> Under what circumstances do nameservers/settings need to be different in
> start vs. run modes?

Haproxy never reads in /etc/resolv.conf; libc does it for us (for libc
based resolution).



> I'd expect that for most installations, the run-time config could/should be
> the same as the start-time config ? Having to create a run-time config that
> will just be the same as the start-time gets in the way of automating of
> config across different environments ...

I can see it wouldn't scale if you have a large number of different
nameserver sets. I guess that is not usually a problem and people have
the same name servers sets or at least provisioning groups using the
same nameserver sets, so automation can handle it in a scalable way.
Or they automate it away in other ways, like with placeholders in
haproxy.cfg and scripts that replace the placeholders locally.

I can certainly see how this would simplify things, but writing a
/etc/resolv.conf parser in userspace is something that I would
consider a specific feature for which someone has to write actual code
for it.


nginx does not parse resolv.conf either, btw:
http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver



Lukas
Jim Freeman
Re: 1.8 resolvers - start vs. run
December 29, 2017 09:10PM
Great feedback - thanks !

I'll take a look at the code ...

On Fri, Dec 29, 2017 at 12:59 PM, Lukas Tribus <[email protected]> wrote:

> Hello,
>
>
> On Fri, Dec 29, 2017 at 7:00 PM, Jim Freeman <[email protected]> wrote:
> > I'm a bit befuddled by the different nameserver config 'twixt these 2
> modes?
> > [ Methinks I grok the need for an internal non-libc/libresolv resolver ]
> >
> > Why isn't the the /etc/resolv.conf start-time config used (or at least
> > available) as a default run-time config (chroot notwithstanding)?
> > Under what circumstances do nameservers/settings need to be different in
> > start vs. run modes?
>
> Haproxy never reads in /etc/resolv.conf; libc does it for us (for libc
> based resolution).
>
>
>
> > I'd expect that for most installations, the run-time config could/should
> be
> > the same as the start-time config ? Having to create a run-time config
> that
> > will just be the same as the start-time gets in the way of automating of
> > config across different environments ...
>
> I can see it wouldn't scale if you have a large number of different
> nameserver sets. I guess that is not usually a problem and people have
> the same name servers sets or at least provisioning groups using the
> same nameserver sets, so automation can handle it in a scalable way.
> Or they automate it away in other ways, like with placeholders in
> haproxy.cfg and scripts that replace the placeholders locally.
>
> I can certainly see how this would simplify things, but writing a
> /etc/resolv.conf parser in userspace is something that I would
> consider a specific feature for which someone has to write actual code
> for it.
>
>
> nginx does not parse resolv.conf either, btw:
> http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
>
>
>
> Lukas
>
Jim Freeman
Re: 1.8 resolvers - start vs. run
December 29, 2017 10:20PM
Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
nameservers [resolv.h], so haproxy won't have to parse it either ...

Will keep poking.



On Fri, Dec 29, 2017 at 12:59 PM, Lukas Tribus <[email protected]> wrote:

> Hello,
>
>
> On Fri, Dec 29, 2017 at 7:00 PM, Jim Freeman <[email protected]> wrote:
> > I'm a bit befuddled by the different nameserver config 'twixt these 2
> modes?
> > [ Methinks I grok the need for an internal non-libc/libresolv resolver ]
> >
> > Why isn't the the /etc/resolv.conf start-time config used (or at least
> > available) as a default run-time config (chroot notwithstanding)?
> > Under what circumstances do nameservers/settings need to be different in
> > start vs. run modes?
>
> Haproxy never reads in /etc/resolv.conf; libc does it for us (for libc
> based resolution).
>
>
>
> > I'd expect that for most installations, the run-time config could/should
> be
> > the same as the start-time config ? Having to create a run-time config
> that
> > will just be the same as the start-time gets in the way of automating of
> > config across different environments ...
>
> I can see it wouldn't scale if you have a large number of different
> nameserver sets. I guess that is not usually a problem and people have
> the same name servers sets or at least provisioning groups using the
> same nameserver sets, so automation can handle it in a scalable way.
> Or they automate it away in other ways, like with placeholders in
> haproxy.cfg and scripts that replace the placeholders locally.
>
> I can certainly see how this would simplify things, but writing a
> /etc/resolv.conf parser in userspace is something that I would
> consider a specific feature for which someone has to write actual code
> for it.
>
>
> nginx does not parse resolv.conf either, btw:
> http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
>
>
>
> Lukas
>
Lukas Tribus
Re: 1.8 resolvers - start vs. run
December 29, 2017 10:30PM
Hi Jim,


On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]> wrote:
> Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
> nameservers [resolv.h], so haproxy won't have to parse it either ...
>
> Will keep poking.

Do give it some time to discuss the implementation here first though,
before you invest a lot of time in a specific direction (especially if
you link to new libraries).

CC'ing Baptise and Willy.



cheers,
lukas
Andrew Smalley
Re: 1.8 resolvers - start vs. run
December 29, 2017 11:00PM
Hello Jim.

I've seen the thread and that you're "befuddled" a little about the use of DNS.,

Think of it this way, with the resolvers in HAProxy you can resolve
the real server names of real server pool, this may be very dynamic in
nature and separate to /etc/resolve.conf

Now imagine a farm of Haproxy servers with different resolves
configured internally, but you want the Haproxy instance to have
public DNS resolved while there may be many split horizon dns
available and maybe not public. Haproxy then ensures it uses the DNS
servers you want it to and not the system resolver

Personally and this is just an opinion I think the Haproxy resolver is
and should be separate to /etc/resolv.conf


Andruw Smalley

Loadbalancer.org Ltd.

www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
asmalley@loadbalancer.org

Leave a Review | Deployment Guides | Blog


On 29 December 2017 at 21:26, Lukas Tribus <[email protected]> wrote:
> Hi Jim,
>
>
> On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]> wrote:
>> Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
>> nameservers [resolv.h], so haproxy won't have to parse it either ...
>>
>> Will keep poking.
>
> Do give it some time to discuss the implementation here first though,
> before you invest a lot of time in a specific direction (especially if
> you link to new libraries).
>
> CC'ing Baptise and Willy.
>
>
>
> cheers,
> lukas
>
Jim Freeman
Re: 1.8 resolvers - start vs. run
December 29, 2017 11:20PM
I'm not proposing use of /etc/resolv.conf *instead* of haproxy's other
configs, only as *a* (default) config [the same default that is good enough
for haproxy to use at start-time].

So if that config suffices (as I suspect it usually does), config is
simplified.

Attached is a trivial program that prints IPv4 nameservers listed in
/etc/resolv.conf (with libresolv doing the parsing).


On Fri, Dec 29, 2017 at 2:56 PM, Andrew Smalley <[email protected]>
wrote:

> Hello Jim.
>
> I've seen the thread and that you're "befuddled" a little about the use of
> DNS.,
>
> Think of it this way, with the resolvers in HAProxy you can resolve
> the real server names of real server pool, this may be very dynamic in
> nature and separate to /etc/resolve.conf
>
> Now imagine a farm of Haproxy servers with different resolves
> configured internally, but you want the Haproxy instance to have
> public DNS resolved while there may be many split horizon dns
> available and maybe not public. Haproxy then ensures it uses the DNS
> servers you want it to and not the system resolver
>
> Personally and this is just an opinion I think the Haproxy resolver is
> and should be separate to /etc/resolv.conf
>
>
> Andruw Smalley
>
> Loadbalancer.org Ltd.
>
> www.loadbalancer.org
> +1 888 867 9504 / +44 (0)330 380 1064
> asmalley@loadbalancer.org
>
> Leave a Review | Deployment Guides | Blog
>
>
> On 29 December 2017 at 21:26, Lukas Tribus <[email protected]> wrote:
> > Hi Jim,
> >
> >
> > On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]> wrote:
> >> Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
> >> nameservers [resolv.h], so haproxy won't have to parse it either ...
> >>
> >> Will keep poking.
> >
> > Do give it some time to discuss the implementation here first though,
> > before you invest a lot of time in a specific direction (especially if
> > you link to new libraries).
> >
> > CC'ing Baptise and Willy.
> >
> >
> >
> > cheers,
> > lukas
> >
>
Jim Freeman
Re: 1.8 resolvers - start vs. run
January 08, 2018 02:00PM
No new libs needed.

libc/libresolv 's res_ninit() suffices ...

http://man7.org/linux/man-pages/man3/resolver.3.html

On Fri, Dec 29, 2017 at 2:26 PM, Lukas Tribus <[email protected]> wrote:

> Hi Jim,
>
>
> On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]> wrote:
> > Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
> > nameservers [resolv.h], so haproxy won't have to parse it either ...
> >
> > Will keep poking.
>
> Do give it some time to discuss the implementation here first though,
> before you invest a lot of time in a specific direction (especially if
> you link to new libraries).
>
> CC'ing Baptise and Willy.
>
>
>
> cheers,
> lukas
>
Baptiste
Re: 1.8 resolvers - start vs. run
January 08, 2018 09:30PM
Hi Jim,

I very welcome this feature. Actually, I wanted to add it myself for some
time now.
I currently work it around using init script, whenever I want to use name
servers provided by resolv.conf.

I propose the following: if no nameserver directives are found in the
resolvers section, then we fallback to resolv.conf parsing.

If you fill comfortable enough, please send me / the ml a patch and I can
review it.
If you have any questions on the design, don't hesitate to ask.

Baptiste


On Mon, Jan 8, 2018 at 1:56 PM, Jim Freeman <[email protected]> wrote:

> No new libs needed.
>
> libc/libresolv 's res_ninit() suffices ...
>
> http://man7.org/linux/man-pages/man3/resolver.3.html
>
> On Fri, Dec 29, 2017 at 2:26 PM, Lukas Tribus <[email protected]> wrote:
>
>> Hi Jim,
>>
>>
>> On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]> wrote:
>> > Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
>> > nameservers [resolv.h], so haproxy won't have to parse it either ...
>> >
>> > Will keep poking.
>>
>> Do give it some time to discuss the implementation here first though,
>> before you invest a lot of time in a specific direction (especially if
>> you link to new libraries).
>>
>> CC'ing Baptise and Willy.
>>
>>
>>
>> cheers,
>> lukas
>>
>
>
Jim Freeman
Re: 1.8 resolvers - start vs. run
January 08, 2018 10:30PM
Your proposal aligns with what I was thinking over the weekend.

I'll try to be clean/small enough to tempt a back-port to 1.8 :-)

On Mon, Jan 8, 2018 at 1:17 PM, Baptiste <[email protected]> wrote:

> Hi Jim,
>
> I very welcome this feature. Actually, I wanted to add it myself for some
> time now.
> I currently work it around using init script, whenever I want to use name
> servers provided by resolv.conf.
>
> I propose the following: if no nameserver directives are found in the
> resolvers section, then we fallback to resolv.conf parsing.
>
> If you fill comfortable enough, please send me / the ml a patch and I can
> review it.
> If you have any questions on the design, don't hesitate to ask.
>
> Baptiste
>
>
> On Mon, Jan 8, 2018 at 1:56 PM, Jim Freeman <[email protected]> wrote:
>
>> No new libs needed.
>>
>> libc/libresolv 's res_ninit() suffices ...
>>
>> http://man7.org/linux/man-pages/man3/resolver.3.html
>>
>> On Fri, Dec 29, 2017 at 2:26 PM, Lukas Tribus <[email protected]> wrote:
>>
>>> Hi Jim,
>>>
>>>
>>> On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]> wrote:
>>> > Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
>>> > nameservers [resolv.h], so haproxy won't have to parse it either ...
>>> >
>>> > Will keep poking.
>>>
>>> Do give it some time to discuss the implementation here first though,
>>> before you invest a lot of time in a specific direction (especially if
>>> you link to new libraries).
>>>
>>> CC'ing Baptise and Willy.
>>>
>>>
>>>
>>> cheers,
>>> lukas
>>>
>>
>>
>
Baptiste
Re: 1.8 resolvers - start vs. run
January 08, 2018 11:30PM
Unfortunately, this may not be backported into 1.8.
We do backport only bug fixes and this is a feature.

Baptiste

On Mon, Jan 8, 2018 at 10:20 PM, Jim Freeman <[email protected]> wrote:

> Your proposal aligns with what I was thinking over the weekend.
>
> I'll try to be clean/small enough to tempt a back-port to 1.8 :-)
>
> On Mon, Jan 8, 2018 at 1:17 PM, Baptiste <[email protected]> wrote:
>
>> Hi Jim,
>>
>> I very welcome this feature. Actually, I wanted to add it myself for some
>> time now.
>> I currently work it around using init script, whenever I want to use name
>> servers provided by resolv.conf.
>>
>> I propose the following: if no nameserver directives are found in the
>> resolvers section, then we fallback to resolv.conf parsing.
>>
>> If you fill comfortable enough, please send me / the ml a patch and I can
>> review it.
>> If you have any questions on the design, don't hesitate to ask.
>>
>> Baptiste
>>
>>
>> On Mon, Jan 8, 2018 at 1:56 PM, Jim Freeman <[email protected]> wrote:
>>
>>> No new libs needed.
>>>
>>> libc/libresolv 's res_ninit() suffices ...
>>>
>>> http://man7.org/linux/man-pages/man3/resolver.3.html
>>>
>>> On Fri, Dec 29, 2017 at 2:26 PM, Lukas Tribus <[email protected]> wrote:
>>>
>>>> Hi Jim,
>>>>
>>>>
>>>> On Fri, Dec 29, 2017 at 10:14 PM, Jim Freeman <[email protected]>
>>>> wrote:
>>>> > Looks like libresolv 's res_ninit() parses out /etc/resolv.conf 's
>>>> > nameservers [resolv.h], so haproxy won't have to parse it either ...
>>>> >
>>>> > Will keep poking.
>>>>
>>>> Do give it some time to discuss the implementation here first though,
>>>> before you invest a lot of time in a specific direction (especially if
>>>> you link to new libraries).
>>>>
>>>> CC'ing Baptise and Willy.
>>>>
>>>>
>>>>
>>>> cheers,
>>>> lukas
>>>>
>>>
>>>
>>
>
Sorry, only registered users may post in this forum.

Click here to login