Re: [paho-dev] MQTT conformance tests independance

[Ian Craggs <icraggs@xxxxxxxxxxxxxxxxxxxxxxx>]

Hi Nicolas,

the tests are meant to be largely independent, but are affected by data left around from previous tests.

In this case, it looks like your retained messages aren't being cleared by sending a zero length message.  The offline message queueing test is being affected by the failure of the retained message test to clear up after itself.

Ian

On 07/17/2015 08:47 PM, Nicolas Jouanin wrote:

Hi,

I’m currently writing a MQTT broker implementation using Python async I/O: HBMQTT [1]. 

I’m using paho conformance test program is order to test it, especially conformance with MQTT spec. Current status is available here [2]. As you can see, some tests are failing and it’s very hard to know why and debugging when all tests are running in sequence. Therefore, in order to fix Offline message queueing test, I’ve modified client_test.py as follow (line 417) in order to only run offline message queuing tests :

tests = [offline_message_queueing_test]

#tests = [basic_test, retained_message_test, offline_message_queueing_test, will_message_test,

#         overlapping_subscriptions_test, keepalive_test, redelivery_on_reconnect_test]

When doing this, the test succeed, while it fails when ran with the whole test suite.

So I wonder if it’s a bug in the test suite (tests are not independent), or if it’s mandatory to run all the tests in sequence ?

[1] : https://github.com/beerfactory/hbmqtt

[2] : http://community.beerfactory.org/jira/browse/HBMQTT-18

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/paho-dev

-- 
Ian Craggs                          
icraggs@xxxxxxxxxx                 IBM United Kingdom
Paho Project Lead; Committer on Mosquitto