Re: [mosquitto-dev] Basic configuration issue with multiple listeners

[Greg Troxel <gdt@xxxxxxxxxx>]

Peter Rockett <p.rockett@xxxxxxxxxxxxxxx> writes:

> I have a Linux machine with three interfaces:

Note that my networking experience is primarily with BSD (since 2.9BSD)
not Linux, but most of this was reimplemented in Linux in a compatible
way.

> 1) localhost (obviously)

To be pedantic, the loopback interface, lo0, configured with 127.0.0.1
and perhaps ::1, together known as localhost.  But yes, I see what you
mean.

I wonder if you have IPv6 configured on lo0.  Mine looks like

  lo0: flags=0x8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33624
          inet 127.0.0.1/8 flags 0x0
          inet6 ::1/128 flags 0x20<NODAD>
          inet6 fe80::1%lo0/64 flags 0x0 scopeid 0x2

but I have no idea if your GNU/Linux distribution views IPv6 as the
normal case.  Probably this isn't related to your problem.

> 2) A network card statically mapped to 192.168.0.1, which is connected
> to wireless access point.
>
> 3) A second network card with a DHCP-leased IP address and connected
> to the public internet.
>
> I want to listen on (1) and (2) for MQTT traffic, but I do not want to
> listen for *any* MQTT traffic on (3).

A reasonable thing to want.

> My mosquitto configuration file (/etc/mosquitto/mosquitto.conf) is the
> default installed version. On top of this, I have a local config file
> in /etc/mosquitto/conf.d that contains the following:
>
> allow_anonymous false
> connection_messages true
> log_dest syslog
> log_type error
> log_type warning
> password_file /etc/mosquitto/passwd
> socket_domain ipv4
> 'A'
>
> where 'A' is some set of directives to configure the listeners. I have
> tried the following:
>
> port 1883
> bind_interface localhost

A major source of difficulty for me is how mosquitto does listener
configuration.   A listener in other configs would be configured as a
stanza that has the listener and the things that control it scoped
somehow.  mosquitto sort of has the listener directive and then the
things that follow modify that listener, until the next  listener.  And
if you have modifier statement before the first new listener, it makes
the default listener be active and then modifies it.   I think this is
basically a bug, but I realize it is hard to change.

> This works fine as verified with mosquitto_sub/pub, plus 'netstat
> -tlpn' lists an LISTEN port associated with the mosquitto process.

The key point is to see if this is bound to *.1883 or 127.0.0.1.1883.
The normal listen binds to INADDR_ANY, which is represented 0.0.0.0, and
matches all addresses.

> From my reading of the docs  & what I *want* to achieve, I would have
> thought that the following should work for me:
>
> listener 1883 localhost
> listener 1883 192.168.0.1
>
> but it does not! Testing with mosquitto_sub reports "Connection
> refused", the log shows the "Error: Address already in use" message 
> on service startup, and netstat lists zero LISTEN ports associated
> with mosquitto. FWIW, the following combinations (tried
> individually!):

EADDRINUSE means that a call to bind() failed, and that can be because
it tried to bind INADDR_ANY first and then a specific address, or
because there were two binds to INADDR_ANY.  Perhaps use some kind of
ktrace/strace/ktruss type program to see what happens.

> port 1883
> bind_interface localhost
> listener 1883 192.168.0.1
>
> listener 1883
>
> listener 1883 localhost
> listener 1885 192.168.0.1
>
> all give exactly the same outcome. Note the last variant using
> different port numbers was suggested by Anil... but same outcome.

Wildly guessing, you might be running into a conflict between the
implicit listener and the one you are trying to configure.  I don't know
how mosquitto behaves on an error like that.

(fuzzy from memory; I could be off) I found, for example, that setting
ca_path, was enough to instantiate the default listener.  A fragment of
a config (that listens on 8883 on all interfaces) is:

  per_listener_settings true
  listener 8883
  protocol mqtt
  dhparamfile /usr/pkg/etc/mosquitto-dhparm.pem
  capath /etc/openssl/certs

after making sure the rest of the config file does not 1) define any
listeneres or 2) have any listener-modifying commands.

> So. Any suggestions on what I am doing wrong here? Or what fundamental
> point I have misunderstood?

You are using the default config which is likely to be trouble for
non-standard listeners.

You're overlooking how hard it is to configure listeners in mosquitto,
and how much you have to be careful to avoid the implicit one.  :-(

So look hard at the default config, and perhaps get rid of it, and
perhaps ask where it came from and why each line is as it is.  The
default config from mosquitto is all comments and blank lines, showing
you what's compiled in.  If your distribution changed that, you should
probably ask them for help (or ask them to explain the rationale for
deviations from upstream).