Hi Ian,

Thanks. That's helpful. And, I also realized that the app and any wrapper libraries  can probably work around most/all the token issues by using userContext pointers.

On 05/20/2013 10:47 AM, Ian Craggs wrote:

   1. call sendMessage()    /* no blocking at all - that is the point of the async API */

I wouldn't think it unreasonable to block long enough to place the message in an outgoing queue. But I understand your point. It does still worry me a little, though, that while waiting to be queued, the messages are lost to the MQTTAsync_getPendingTokens call.

   2. get the token back in the onSuccess callback /* admittedly I don't like the timing of this callback currently which is at the same time as deliveryComplete.  I was trying to match the _javascript_ API.  I think this should happen at the same time as does the return from MQTTClient API (see below) which would give you the information to use in getPendingTokens, as soon as it's available.  */

When you say "at the same time" can you guarantee that onSuccess is called first? Serially? Will onSuccess return before deliveryComplete is called (meaning are they called by the same thread, or by different ones?)

I always thought waitForCompletion() was an erroneous and unnecessary addition to the Java *async* API.  

Why use the asynchronous API if you don't want to use callbacks?

Well it depends on your application. Certainly a GUI may never want to block, but a microcontroller might send a message, do something else for a while, and then rendezvous/wait on the message confirmation. In that case I think what you may tend to find is that in practice a large percentage of callbacks become:

volatile bool done = false;
void myCallback() { done = true; }

So the callbacks are completely powerful and flexible, but quite often you just want simplicity.

So I think the trick to a great library is in reducing the amount of similar code that different users will be forced to recreate. That's my biggest problem with the standard C library. I mean, seriously, how many times do I have to rewrite a linked list?!?!

Frank

On 20/05/13 15:01, Frank Pagliughi wrote:

Hey Ian,

It presents a simpler means of synchronization, allows the client to block in an OS-independent way, and mimics the API of the MQTT client libraries in other languages. A wait() function as the sole blocking call is a common pattern in other modern asynchronous libraries: Microsoft Async, C++11 threads/futures. the Paho Java client library, etc. I'm not a GUI guy, I'm an RTOS guy, and do this pattern a lot, where a completed action always signals an OS synchronization object, but also fires an optional callback if the user requested it.

In particular, as I originally mentioned, I'm trying to make the C++ API mimic the Java one as much as possible, like:

mqtt::itoken tok = client.publish("some topic", myMessage);
// Do something else for a while
tok.wait_for_completion();

I can work around this in the C++ library by intercepting the callbacks, but two issues make this a little more complicated due to to the outgoing "send" thread in the C lib:

(1) There doesn't appear to be a way to get the MQTTAsync_token of a message when you send it. And thus it's difficult to tie a token to the message.

(2) Upon return of the send/sendMessage calls, the message has "disappeared" into the queue of the send thread. It doesn't appear in the list of outgoing messages (MQTTAsync_getPendingTokens) until some time later. So it is easy for the app to think that all messages were sent when some have yet to even be queued for transmission.

Thanks for your time,
Frank

On 05/20/2013 05:30 AM, Ian Craggs wrote:

A deliberate choice.  Why have one blocking call in an asynchronous API which is designed for GUI use, amongst others?  This is an event-oriented API - use the deliveryComplete callback.  To receive messages with MQTTAsync, you have to use callbacks.  You have to use callbacks to determine the success of API calls. 

For a blocking, simpler to use API, with less use of callbacks, there is MQTTClient.

Ian


On 19/05/13 19:06, Frank Pagliughi wrote:

Well, while I'm throwing sand in the works, I might as well continue...

One thing that appears to be "missing" from the C async API is the ability to block a thread on a token while waiting for an action to complete. Something like:

int MQTTAsync_waitForCompletion(MQTTAsync handle, MQTTAsync_token token, int timeout);

This would prevent the app from wasting CPU cycles while spinning on a flag from a callback, and can simplify client apps that don't need the full power of a callback function by eliminating the need for them for basic synchronization. Plus it would allow the user to write more portable client apps, since the library would hide the OS-specific thread functionality.

It appears that the library already wraps condition variables in Linux. Maybe Windows could use a Manual Reset Event to do this?

Frank

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/paho-dev

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/paho-dev

   1. call sendMessage()    /* no blocking at all - that is the point of the async API */

   2. get the token back in the onSuccess callback /* admittedly I don't like the timing of this callback currently which is at the same time as deliveryComplete.  I was trying to match the _javascript_ API.  I think this should happen at the same time as does the return from MQTTClient API (see below) which would give you the information to use in getPendingTokens, as soon as it's available.  */

I always thought waitForCompletion() was an erroneous and unnecessary addition to the Java *async* API.  

Why use the asynchronous API if you don't want to use callbacks?

volatile bool done = false;
void myCallback() { done = true; }

On 20/05/13 15:01, Frank Pagliughi wrote:

Hey Ian,

It presents a simpler means of synchronization, allows the client to block in an OS-independent way, and mimics the API of the MQTT client libraries in other languages. A wait() function as the sole blocking call is a common pattern in other modern asynchronous libraries: Microsoft Async, C++11 threads/futures. the Paho Java client library, etc. I'm not a GUI guy, I'm an RTOS guy, and do this pattern a lot, where a completed action always signals an OS synchronization object, but also fires an optional callback if the user requested it.

In particular, as I originally mentioned, I'm trying to make the C++ API mimic the Java one as much as possible, like:

mqtt::itoken tok = client.publish("some topic", myMessage);
// Do something else for a while
tok.wait_for_completion();

I can work around this in the C++ library by intercepting the callbacks, but two issues make this a little more complicated due to to the outgoing "send" thread in the C lib:

(1) There doesn't appear to be a way to get the MQTTAsync_token of a message when you send it. And thus it's difficult to tie a token to the message.

(2) Upon return of the send/sendMessage calls, the message has "disappeared" into the queue of the send thread. It doesn't appear in the list of outgoing messages (MQTTAsync_getPendingTokens) until some time later. So it is easy for the app to think that all messages were sent when some have yet to even be queued for transmission.

Thanks for your time,
Frank

On 05/20/2013 05:30 AM, Ian Craggs wrote:

A deliberate choice.  Why have one blocking call in an asynchronous API which is designed for GUI use, amongst others?  This is an event-oriented API - use the deliveryComplete callback.  To receive messages with MQTTAsync, you have to use callbacks.  You have to use callbacks to determine the success of API calls. 

For a blocking, simpler to use API, with less use of callbacks, there is MQTTClient.

Ian


On 19/05/13 19:06, Frank Pagliughi wrote:

Well, while I'm throwing sand in the works, I might as well continue...

One thing that appears to be "missing" from the C async API is the ability to block a thread on a token while waiting for an action to complete. Something like:

int MQTTAsync_waitForCompletion(MQTTAsync handle, MQTTAsync_token token, int timeout);

This would prevent the app from wasting CPU cycles while spinning on a flag from a callback, and can simplify client apps that don't need the full power of a callback function by eliminating the need for them for basic synchronization. Plus it would allow the user to write more portable client apps, since the library would hide the OS-specific thread functionality.

It appears that the library already wraps condition variables in Linux. Maybe Windows could use a Manual Reset Event to do this?

Frank

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/paho-dev

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/paho-dev

Hey Ian,

It presents a simpler means of synchronization, allows the client to block in an OS-independent way, and mimics the API of the MQTT client libraries in other languages. A wait() function as the sole blocking call is a common pattern in other modern asynchronous libraries: Microsoft Async, C++11 threads/futures. the Paho Java client library, etc. I'm not a GUI guy, I'm an RTOS guy, and do this pattern a lot, where a completed action always signals an OS synchronization object, but also fires an optional callback if the user requested it.

In particular, as I originally mentioned, I'm trying to make the C++ API mimic the Java one as much as possible, like:

mqtt::itoken tok = client.publish("some topic", myMessage);
// Do something else for a while
tok.wait_for_completion();

I can work around this in the C++ library by intercepting the callbacks, but two issues make this a little more complicated due to to the outgoing "send" thread in the C lib:

(1) There doesn't appear to be a way to get the MQTTAsync_token of a message when you send it. And thus it's difficult to tie a token to the message.

(2) Upon return of the send/sendMessage calls, the message has "disappeared" into the queue of the send thread. It doesn't appear in the list of outgoing messages (MQTTAsync_getPendingTokens) until some time later. So it is easy for the app to think that all messages were sent when some have yet to even be queued for transmission.

Thanks for your time,
Frank

On 05/20/2013 05:30 AM, Ian Craggs wrote:

A deliberate choice.  Why have one blocking call in an asynchronous API which is designed for GUI use, amongst others?  This is an event-oriented API - use the deliveryComplete callback.  To receive messages with MQTTAsync, you have to use callbacks.  You have to use callbacks to determine the success of API calls. 

For a blocking, simpler to use API, with less use of callbacks, there is MQTTClient.

Ian


On 19/05/13 19:06, Frank Pagliughi wrote:

Well, while I'm throwing sand in the works, I might as well continue...

One thing that appears to be "missing" from the C async API is the ability to block a thread on a token while waiting for an action to complete. Something like:

int MQTTAsync_waitForCompletion(MQTTAsync handle, MQTTAsync_token token, int timeout);

This would prevent the app from wasting CPU cycles while spinning on a flag from a callback, and can simplify client apps that don't need the full power of a callback function by eliminating the need for them for basic synchronization. Plus it would allow the user to write more portable client apps, since the library would hide the OS-specific thread functionality.

It appears that the library already wraps condition variables in Linux. Maybe Windows could use a Manual Reset Event to do this?

Frank

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/paho-dev