[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
[paho-dev] MQTT Conformance/Interoperability Testing status
|
Hi,
I was wondering what was the current status of broker testing suite. I mean is it currently reliable for a broker compliance testing ?
I’d like to use it to validate my own broker implementation, but some tests are failing. Also I’ve tried to run the conformance tests on test.mosquito.org and it also fails on some test (see results below).
python3 client_test.py --hostname=test.mosquitto.org -p 1883
hostname test.mosquitto.org port 1883
clean up starting
ERROR:root:receive: unexpected exception (<class 'UnicodeDecodeError'>, UnicodeDecodeError('utf-8', b'/v1/p0/atmeld/aeef/stat\x9d', 23, 24, 'invalid start byte'), <traceback object at 0x104826208>)
Traceback (most recent call last):
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/client/internal.py", line 44, in receive
packet = MQTTV3.unpackPacket(MQTTV3.getPacket(self.socket))
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/formats/MQTTV311.py", line 892, in unpackPacket
packet.unpack(buffer)
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/formats/MQTTV311.py", line 460, in unpack
self.topicName = readUTF(buffer[fhlen:], packlen - curlen)
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/formats/MQTTV311.py", line 178, in readUTF
buf = buffer[2:2+length].decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9d in position 23: invalid start byte
ERROR:root:call: unexpected exception (<class 'UnicodeDecodeError'>, UnicodeDecodeError('utf-8', b'/v1/p0/atmeld/aeef/stat\x9d', 23, 24, 'invalid start byte'), <traceback object at 0x104826248>)
Traceback (most recent call last):
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/client/internal.py", line 155, in __call__
self.receive(callback)
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/client/internal.py", line 44, in receive
packet = MQTTV3.unpackPacket(MQTTV3.getPacket(self.socket))
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/formats/MQTTV311.py", line 892, in unpackPacket
packet.unpack(buffer)
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/formats/MQTTV311.py", line 460, in unpack
self.topicName = readUTF(buffer[fhlen:], packlen - curlen)
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/formats/MQTTV311.py", line 178, in readUTF
buf = buffer[2:2+length].decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9d in position 23: invalid start byte
deleting retained message for topic frl/2130
deleting retained message for topic /ohotkuba/19791981jh
deleting retained message for topic /ohotkuba/19791981ih
deleting retained message for topic /casa/corredor/luz
deleting retained message for topic /casa/sala/controlador/luz
deleting retained message for topic /casa/sala/luz/cor
deleting retained message for topic /casa/sala/slider
deleting retained message for topic /casa/jardim/luz
deleting retained message for topic /casa/garagem/luz
deleting retained message for topic /Schlenz/AndroidCI
deleting retained message for topic /Schlenz/WebCI
deleting retained message for topic /Schlenz/iOSCI
deleting retained message for topic /jolicode/coffee_pot
deleting retained message for topic /node0/dht0/temperature
deleting retained message for topic /node0/dht0/humidity
deleting retained message for topic /node0/gas0/pollution
deleting retained message for topic /dcc/jodo
deleting retained message for topic /_mytopic/date
deleting retained message for topic /openhab/in/Date/state
deleting retained message for topic /openhab/in/Weather_Temperature/state
deleting retained message for topic /openhab/in/Light_GF_Living_Table/state
deleting retained message for topic /openhab/in/Light_GF_Corridor_Ceiling/state
deleting retained message for topic /openhab/in/Light_GF_Kitchen_Ceiling/state
deleting retained message for topic /openhab/in/Light_GF_Kitchen_Table/state
deleting retained message for topic /openhab/in/Light_GF_Corridor_Wardrobe/state
deleting retained message for topic /openhab/in/Light_GF_Toilet_Ceiling/state
deleting retained message for topic /openhab/in/Light_GF_Toilet_Mirror/state
deleting retained message for topic /openhab/in/Light_FF_Bath_Ceiling/state
deleting retained message for topic /openhab/in/Light_FF_Bath_Mirror/state
deleting retained message for topic /openhab/in/Light_FF_Corridor_Ceiling/state
deleting retained message for topic /openhab/in/Light_FF_Office_Ceiling/state
deleting retained message for topic /openhab/in/Light_FF_Child_Ceiling/state
deleting retained message for topic /openhab/in/Light_FF_Bed_Ceiling/state
deleting retained message for topic /openhab/in/Light_C_Corridor_Ceiling/state
deleting retained message for topic /openhab/in/Light_C_Staircase/state
deleting retained message for topic /openhab/in/Light_C_Washing_Ceiling/state
deleting retained message for topic /openhab/in/Light_C_Workshop/state
deleting retained message for topic /openhab/in/Light_Outdoor_Garage/state
deleting retained message for topic /openhab/in/Light_Outdoor_Terrace/state
deleting retained message for topic /openhab/in/Light_Outdoor_Frontdoor/state
deleting retained message for topic /openhab/in/Heating_GF_Corridor/state
deleting retained message for topic /openhab/in/Heating_GF_Toilet/state
deleting retained message for topic /openhab/in/Heating_GF_Living/state
deleting retained message for topic /openhab/in/Heating_GF_Kitchen/state
deleting retained message for topic /openhab/in/Heating_FF_Bath/state
deleting retained message for topic /openhab/in/Heating_FF_Office/state
deleting retained message for topic /openhab/in/Heating_FF_Child/state
deleting retained message for topic /openhab/in/Heating_FF_Bed/state
deleting retained message for topic /openhab/in/Temperature_Setpoint/state
deleting retained message for topic /openhab/in/Window_GF_Frontdoor/state
deleting retained message for topic /openhab/in/Window_GF_Kitchen/state
deleting retained message for topic /openhab/in/Window_GF_Living/state
deleting retained message for topic /openhab/in/Window_GF_Toilet/state
deleting retained message for topic /openhab/in/Window_FF_Bath/state
deleting retained message for topic /openhab/in/Window_FF_Bed/state
deleting retained message for topic /openhab/in/Window_FF_Office_Window/state
deleting retained message for topic /openhab/in/Window_FF_Office_Door/state
deleting retained message for topic /openhab/in/Garage_Door/state
deleting retained message for topic /openhab/in/Temperature_GF_Corridor/state
deleting retained message for topic /openhab/in/Temperature_GF_Toilet/state
deleting retained message for topic /openhab/in/Temperature_GF_Living/state
deleting retained message for topic /openhab/in/Temperature_GF_Kitchen/state
deleting retained message for topic /openhab/in/Temperature_FF_Bath/state
deleting retained message for topic /openhab/in/Temperature_FF_Office/state
deleting retained message for topic /openhab/in/Temperature_FF_Child/state
deleting retained message for topic /openhab/in/Temperature_FF_Bed/state
deleting retained message for topic /openhab/in/Weather_Temp_Max/state
deleting retained message for topic /openhab/in/Weather_Temp_Min/state
deleting retained message for topic /home
deleting retained message for topic /home/temperature
deleting retained message for topic /openHABBruegge/out/Slim_motion/state
deleting retained message for topic /openHABBruegge/out/Slim_temp/state
deleting retained message for topic /openHABBruegge/out/Slim_door/state
deleting retained message for topic /openHABBruegge/out/Slim_battery/state
deleting retained message for topic /v1/p0/starling/3810/status
deleting retained message for topic /v1/p0/buyAtmel/aeef/status
deleting retained message for topic /v1/p0/atmeld/aeef/status
clean up finished
Basic test starting
Basic test succeeded
Retained message test starting
Traceback (most recent call last):
File "client_test.py", line 142, in retained_message_test
aclient.disconnect()
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/client/main.py", line 187, in disconnect
assert self.__receiver.inMsgs == {}, self.__receiver.inMsgs
AssertionError: {47: Publishes(DUP=False, QoS=2, Retain=True, MsgId=47, TopicName='PREFIX/DeviceList', Payload=b'{\n "2,1" : {\n "Description" : "N/A",\n "Manufacturer" : "Sogeti HT",\n "Name" : "SignalGenerator"\n }\n}\n'), 48: Publishes(DUP=False, QoS=2, Retain=True, MsgId=48, TopicName='SETest/DeviceList', Payload=b'{\n "6,1" : {\n "Description" : "N/A",\n "Manufacturer" : "Sogeti HT",\n "Name" : "SignalGenerator"\n }\n}\n'), 51: Publishes(DUP=False, QoS=2, Retain=True, MsgId=51, TopicName='hello/world', Payload=b'yooooooooooooooooooo'), 52: Publishes(DUP=False, QoS=2, Retain=True, MsgId=52, TopicName='hello/emqtt', Payload=b'Example: QoS 2'), 53: Publishes(DUP=False, QoS=2, Retain=True, MsgId=53, TopicName='hello/1234', Payload=b'rep'), 54: Publishes(DUP=False, QoS=2, Retain=True, MsgId=54, TopicName='hello/1126', Payload=b'7c:dd:90:78:51:23'), 55: Publishes(DUP=False, QoS=2, Retain=True, MsgId=55, TopicName='hello/27137', Payload=b'7c:dd:90:78:51:23'), 56: Publishes(DUP=False, QoS=2, Retain=True, MsgId=56, TopicName='hello/4457', Payload=b'7c:dd:90:78:51:23'), 57: Publishes(DUP=False, QoS=2, Retain=True, MsgId=57, TopicName='hello/21670', Payload=b'7c:dd:90:78:51:23'), 58: Publishes(DUP=False, QoS=2, Retain=True, MsgId=58, TopicName='hello/21622', Payload=b'7c:dd:90:78:51:23')}
Retained message test failed
Traceback (most recent call last):
File "client_test.py", line 218, in offline_message_queueing_test
aclient.disconnect()
File "/Users/nico/Dev/beerfactory/org.eclipse.paho.mqtt.testing/interoperability/mqtt/client/main.py", line 187, in disconnect
assert self.__receiver.inMsgs == {}, self.__receiver.inMsgs
AssertionError: {47: Publishes(DUP=False, QoS=2, Retain=True, MsgId=47, TopicName='PREFIX/DeviceList', Payload=b'{\n "2,1" : {\n "Description" : "N/A",\n "Manufacturer" : "Sogeti HT",\n "Name" : "SignalGenerator"\n }\n}\n'), 48: Publishes(DUP=False, QoS=2, Retain=True, MsgId=48, TopicName='SETest/DeviceList', Payload=b'{\n "6,1" : {\n "Description" : "N/A",\n "Manufacturer" : "Sogeti HT",\n "Name" : "SignalGenerator"\n }\n}\n'), 51: Publishes(DUP=False, QoS=2, Retain=True, MsgId=51, TopicName='hello/world', Payload=b'yooooooooooooooooooo'), 52: Publishes(DUP=False, QoS=2, Retain=True, MsgId=52, TopicName='hello/emqtt', Payload=b'Example: QoS 2'), 53: Publishes(DUP=False, QoS=2, Retain=True, MsgId=53, TopicName='hello/1234', Payload=b'rep'), 54: Publishes(DUP=False, QoS=2, Retain=True, MsgId=54, TopicName='hello/1126', Payload=b'7c:dd:90:78:51:23'), 55: Publishes(DUP=False, QoS=2, Retain=True, MsgId=55, TopicName='hello/27137', Payload=b'7c:dd:90:78:51:23'), 56: Publishes(DUP=False, QoS=2, Retain=True, MsgId=56, TopicName='hello/4457', Payload=b'7c:dd:90:78:51:23'), 57: Publishes(DUP=False, QoS=2, Retain=True, MsgId=57, TopicName='hello/21670', Payload=b'7c:dd:90:78:51:23'), 58: Publishes(DUP=False, QoS=2, Retain=True, MsgId=58, TopicName='hello/21622', Payload=b'7c:dd:90:78:51:23')}
Offline message queueing test failed
Traceback (most recent call last):
File "client_test.py", line 178, in will_message_test
assert len(callback2.messages) == 1, callback2.messages # should have the will message
AssertionError: [('Topic/C', b'qos 1', 1, True, 1), ('Topic/C', b'client not disconnected', 2, False, 2)]
Will message test failed
Overlapping subscriptions test starting
Traceback (most recent call last):
File "client_test.py", line 253, in overlapping_subscriptions_test
assert len(callback.messages) in [1, 2]
AssertionError
Overlapping subscriptions test failed
Keepalive test starting
Keepalive test succeeded
Redelivery on reconnect test starting
Traceback (most recent call last):
File "client_test.py", line 309, in redelivery_on_reconnect_test
assert len(callback2.messages) == 2, "length should be 2: %s" % callback2.messages
AssertionError: length should be 2: [('TopicA/B', b'', 1, False, 2), ('TopicA/C', b'qos 2', 2, True, 1), ('TopicA/C', b'', 2, False, 3)]
Redelivery on reconnect test failed
test suite failed
