[mosquitto-dev] user experience report trying to configure a bridge

[Greg Troxel <gdt@xxxxxxxxxx>]

background:
I am vaguely familiar with the concept of MQTT bridging, but have never
tried to configure one.  I successfully have a mosquitto broker running
with TLS on one machine.  My goal is to set up a second broker and to
bridge (speaking loosely to humans) "sensors/#" in and out.  For
context, I have a long background in network protocols, including design
and implementation, and I understand the PKI world.  This note describes
my difficulties.  I'm not meaning to sound cranky -- more to list areas
for improvement,

This is all with 1.6.7.

First, the bridge section starts off with "addresses", but if you put
that first you get a hard-to-understand error.  Later on you find
"connection", and guess that really it has to come first, because
there's the same sort of hard-to-follow pseudo-hierarchy with bridges
that there is with listeners.  This is a bug in the config file syntax,
and  the man page should explain this clearly
and loudly, saying that you start with connection, and then directives
apply to that connection until some point when they don't -- which is
still unclear to me.

The connection statement says the name is used as client id on the
remote.  I don't follow this, as I would expect the bridge locally to be
named by the name of the remote broker, but when connecting one should
be presenting something that has the local broker's name.   The
local_clientid seems to default to something odd; the remote_clientid
default seems to presume a naming scheme that makes sense once reverse
engineered from the defaults.

The addresses section talks about "addresses", but it doesn't explain
about name (to be looked up in DNS) vs address.  It doesn't explain how
one types a v6 address, which has a :, also used as a separator.  It
doesn't explain what happens if one gives a name which has both a v4 and
a v6 address (which really should be the normal case these days, even if
mosquitto doesn't default to listening on v4 and v6!).

The "local" and "remote" language is confusing. In particular,
"local_clientid" is hard to understand.  It seems to be about a notion
of two-way logical communication over the connection made from the local
broker to the bridge, and the clientid assigned to the remote side for
the inbound logical connection.  Rereading the man page, I see that
there is a notion of a configured bridge being treated like an incoming
connection, wtih the crednetials for it given in he config file.  Once
explained, this makes sense (given enough background where one could see
how non-bridged MQTT would be extended, given a constraint of allowing
"topic in" on bridges because prrobably NAT-impaired brokers have to
connect to brokers with a working IP address), but the concept is not
explained up front.

With a numeric address in "address", I get an ssl verify error,
presumably because the address given is not the CN or one of the SANs of
the certificate.  It then prints "socket error", implying an OS-level
error but I think it's just the SSL verification failure (which is not
really, a "socket" error).

With a name in address as in
  address foo.example.com:8123
I see "Socket error on client local.MY_CONNECTION_NAME, disconnecting.",
without any openssl errors.   I don't see any packets either.   So I am
guessing that address  really nees a literal IP address and there is no
name resolution, but that seems hard to believe.

The error is hard to follow in two ways.  One, it's not clear which
socket (the socket used to call connect() on to get a TCP connection to
the remote broker?), and two presumably some system call returned -1
with some errno, and I don't understand why that information isn't
provided.

It's also strange to see "local." as part of a socket, and the man page
did not explain that there is a pseudo-socket from the point of view of
the broker which is handled by the bridging code - if I'm guessing
correctly here.

With TLS, and the apparent requirement for an IP address, I don't
understand how the certificate verification works for anyone, assuming
normal practice of CN with name, and not putting the IP address as a
SAN.

The "bridge_insecure" option has a strange name.  This is about skipping
the name verification check in TLS (and presumably only that).  So it
should be bridge_verify_name, defaulting to true, or something like
that.  "secure" is a word that has a vast number of apsects, and the
name doesn't help understanding.  It would be nice if one could give
"bridge_name_accept foo" to allow the connection if the remote presents
a valid certificate for foo.  That would let people paper over problems
in a narrow way, rather than skipping name verification entirely.

There aren't hints about local and remote clientid and whether they
should have matching names, or if bridges should be named for both
endpoints (vs naming it on the initiating end for the remote end).

After setting client ids for local and remote, and remote user/pass
(matching password file on remote), and local user/pass (matching local
acl file), I managed to get the connection to come up, but then I get a
"socket error' on the local socket

Sep 26 20:24:12 local mosquitto[13174]: Connecting bridge remote.example.com (1.2.3.4:8123)
Sep 26 20:24:13 local mosquitto[13174]: Client bridge.remote-local disconnected due to protocol error.

I'm guessing that this is because I am not doing TLS from my local
bridge endpoint to my broker, even though it's all code in the same
address space.  In fact, I'm mystified about password, becuase a
configured username in the config file should be believed.

try_private is a confusing name.   It seems to be grounded in the spec,
but it's not clear, and it's not clear what "private" means in this context.

I would be interested to see an example working bridge config, or a
pointer to a description; it feels like I am having to figure out many
things more than I would expect.

Thanks,
Greg