Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [mosquitto-dev] Determine client ID from DB
Hi all, thanks for input.
A small script which shows clients and topics sorted by number of messages. Useful to find clients which are requesting persistent connections (cleansession=false) but disconnect after subscribing:

#!/bin/bash


mosquitto_db_dump /var/lib/mosquitto/mosquitto.db > mdump.txt

cat  mdump.txt |grep -A 10 'DB_CHUNK_MSG_STORE:'|egrep 'Topic|Store ID'|tr -d '\n' |sed -e 's/Store/\nStore/g' > storeTopic.txt

echo ""
echo "Largest topics:"
cat storeTopic.txt | grep Topic | sed 's/.*Topic//'|uniq -c | sort -g|tail
echo ""


\rm brokerdata.db

echo  'create table storetopic (storeid,topic);'|sqlite3 brokerdata.db
echo "BEGIN;" > tmp1.sql
cat storeTopic.txt | awk '{printf("insert into storetopic(storeid,topic) values ('"'"'%s'"'"','"'"'%s'"'"');\n", $3,$5)}'   >> tmp1.sql
echo "COMMIT;" >> tmp1.sql
cat tmp1.sql | sqlite3 brokerdata.db


cat mdump.txt |grep -A 9 'DB_CHUNK_CLIENT_MSG:'| tr -d '\n'| sed -e 's/DB/\nDB/g' > clientmsg.txt

echo  'create table clientmsg (clientid,storeid);'|sqlite3 brokerdata.db
echo "BEGIN;" > tmp1.sql
cat clientmsg.txt | awk '{printf("insert into clientmsg(clientid,storeid) values ('"'"'%s'"'"','"'"'%s'"'"');\n", $6,$9)}'   >> tmp1.sql
echo "COMMIT;" >> tmp1.sql
cat tmp1.sql | sqlite3 brokerdata.db


echo "Messages stored in topics probably from persistent clients no longer listening:"
echo 'select topic,clientid from storetopic,clientmsg  where storetopic.storeid = clientmsg.storeid;'|sqlite3 brokerdata.db |sort | uniq -c |sort -g



 

Spiros Ioannou 
Technical Manager, IT/SM
inAccess
www.inaccess.com

M: +30 6973-903808
W: +30 210-6802-358


On 11 December 2017 at 16:58, jianhui zhan <hui6075@xxxxxxxxxxx> wrote:

sorry I was wrong, I miss a line of code, previous client would be disconnect immediately, whatever new clean session is false or true.

btw, these is no topic stored for a subscriber, the topic inside SUBSCRIBE msg would be free after add this subscriber to the sub-tree.

From: jianhui zhan
Sent: Monday, December 11, 2017 10:37:45 PM

To: General development discussions for the mosquitto project
Subject: Re: [mosquitto-dev] Determine client ID from DB
 

Hi,


From current implementation, I saw that while receive a client id with changes clean session = false to true, the Mosquitto would not disconnect/clean previous client, until a periodical persistent client check, or never. This depends on the "persistent_client_expiration" which wrote inside your mosquitto.conf.



From: mosquitto-dev-bounces@eclipse.org <mosquitto-dev-bounces@eclipse.org> on behalf of Mahesh Herle <mahesh@xxxxxxxxxx>
Sent: Monday, December 11, 2017 17:17
To: General development discussions for the mosquitto project
Subject: Re: [mosquitto-dev] Determine client ID from DB
 
The mosquito dB dump program also publishes the topic subscription (which dumps, topic name, QoS, clientid and timestamp I think, in addition to other details).

We had used it sometime to look for all clients of a particular kind and get the clientid so that we can remove the subscription of theirs (by running the mosquitto_sub command with -I option)

On 11-Dec-2017 2:41 PM, "Spiros Ioannou" <sivann@xxxxxxxxxxxx> wrote:
Hi,
we switched our clients from cleansession=false to true, but a topic still seems to store max_queued_messages, we verified with mosquitto_db_dump. 

Is there a way to determine the client_id that was used to subscribe to this topic ? From the db dump, only the "Source ID" client_id is recognizable, which corresponds to the publisher not the subscriber, I presume client_ids of the subscribers are there somewhere, to be used by the broker to give subscribers the data on reconnection.


More info about our setup:
A "server" reads publishes by multiple "clients" in multiple topics. A special topic (the one in question) is named "keepalive"; and all clients suscribe to it. the "server" publishes timestamps there, and those timestamps are received the remote clients and used to throttle or stop their own publishes to other topics in case the server goes down to avoid filling up max_queued_messages.
 

Best Regards,
-Spiros

Spiros Ioannou 
Technical Manager, IT/SM
inAccess
www.inaccess.com

M: +30 6973-903808
W: +30 210-6802-358


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any views expressed by the sender in this communication are those of the sender alone and do not necessarily express the views or the policy of the organization unless intended to do so. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. If you have received this email in error please immediately notify admin@xxxxxxxxxx and delete this email from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Thank you.

_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

Back to the top