Re: [paho-dev] MQTT Server Setup for Testing of MQTT clients

[Roger Light <roger@xxxxxxxxxx>]

Hi Ian,

I think we're in agreement.

I don't automate large message tests at the moment, but I don't see
that writing a test to do so would be tricky. Irregardless, there are
definitely situations where using a broker/client library at the other
end of the test make sense.

One key point in favour of the approach I have taken is the ability to
produce unexpected behaviour and confirm that the broker/client
library responds as intended. Dropping messages to see if they are
retried, invalid packets, that sort of thing.

Cheers,

Roger

On Mon, Jul 22, 2013 at 3:44 PM, Ian Craggs
<icraggs@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
> Thanks Roger, I'll take a look.
>
> I think I'd still want to run some client tests against a "real" server
> though, and vice versa.  Getting enough coverage with your technique seems
> likely to be to be more time-consuming than using a server we have already
> built.   Why not use it, if we have it? What about "large" message tests,
> for instance, where the packet has to be written in more than one attempt
> (for clients/servers that don't have one thread per client)?
>
> I'll think about it.
>
> Ian
>
>
> On 07/22/2013 02:19 PM, Roger Light wrote:
>>
>> Hi Ian,
>>
>> For the mosquitto tests I run a per test instance of the broker for
>> exactly the reasons you say - many of the tests require special
>> configuration. Likewise, I think that running the broker on the build
>> machine is a sensible option because it is not affected by
>> connectivity problems or by whatever else is happening on the server.
>>
>> For both the broker and the client, I don't use the opposite
>> client/broker at all, instead generating and examining received
>> packets within the test itself. I find it much easier to verify what
>> the test is doing when a client test doesn't depend on the broker and
>> vice versa. I suppose for completeness I should be testing them both
>> together as well as these isolated tests.
>>
>> You can view my client tests already because they are part of the Paho
>> Python repository - see the test/lib directory. I would already have
>> them running on eclipse hudson, but I need to request some more recent
>> versions of Python be installed on the build machines.
>>
>> Cheers,
>>
>> Roger
>>
>> On Mon, Jul 22, 2013 at 12:56 PM, Ian Craggs
>> <icraggs@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
>>>
>>> Hi All,
>>>
>>> Al and I have been adding tests for the Java and C clients, which are run
>>> by
>>> Hudson build jobs (see
>>> https://hudson.eclipse.org/hudson/job/paho-c-dev-nightly/ for the C job).
>>>
>>> For the basic MQTT tests, we have pointed the tests at m2m.eclipse.org,
>>> which works fine.  But now we need to add SSL and failover tests (for
>>> High
>>> Availability configurations), so we need to have MQTT server setups to
>>> test
>>> against.
>>>
>>> We could, along with m2m.eclipse.org, have SSL and HA predefined setups
>>> which can be used by the tests.  Would m2m.eclipse.org be the right
>>> place,
>>> or should we have specific test servers somewhere else?
>>>
>>> Or, once we had Mosquitto contributed to Eclipse, I thought the builds
>>> for
>>> the clients and server could use each other for testing.  Then the tests
>>> could configure the server as they see fit, for that test.  The simplest
>>> setup would entail running the MQTT server on the build machine where the
>>> clients were being built.
>>>
>>> Thoughts, suggestions?
>>>
>>> Ian
>>>
>>>
>>> _______________________________________________
>>> paho-dev mailing list
>>> paho-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/paho-dev
>>>
>> _______________________________________________
>> paho-dev mailing list
>> paho-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/paho-dev
>
>
> _______________________________________________
> paho-dev mailing list
> paho-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/paho-dev