Community
Participate
Working Groups
Non open source version of the RSE client cannot connect to the open source version of the RSE server. The requirement is to allow the old client connect to the new server.
This is profoundly an IBM thing to address. Just two words of warning though: 1.) Maintaining backward compatibility in data transfer protocols is not an easy thing to do, and potentially multiplies the testing effort by the number of old protocols to support. 2.) The default daemon port has changed for openRSE, so it's possible to run both an openRSE daemon and an ibmRSE daemon on the same machine.
Right now the only reason the old client fails to connect with a new server is because the client checks the server version, determining that the versions are too far apart. If the client's version check were able to permit 2 versions back (as the new client permits of the server), then this would not be a problem. But to do that requires a change to the old client (which I'm not sure would be acceptable for this product). Another approach would be to move the open rse datastore version back (i.e. from 9.0.0 to 8.0.0). These aren't the most appealing of choices but I'm not sure how else to deal with this - any ideas?
Hm. Moving back the openRSE server version, openRSE clients would no longer be able to connect? Also, behavior of the openRSE server changed. Consider the new "cancel" ability for archive operations, for instance. How would that work when the openRSE server claims it is the old server? Client would be unable to tell the difference.
(In reply to comment #3) > Hm. Moving back the openRSE server version, openRSE clients would no longer be > able to connect? I think it might be okay because dstore accepts servers that are 1 or 2 versions behind, although we show warning messages. > Also, behavior of the openRSE server changed. Consider the new "cancel" ability > for archive operations, for instance. How would that work when the openRSE > server claims it is the old server? Client would be unable to tell the > difference. I think the client just needs to check if there's an associated command descriptor for a command (i.e. 'cancel') in order to determine whether it can do the operation. The miners determine which command descriptors are available. In these cases, it's not the dstore version that matters, it's the datastore schema which is synched on both the client and server side. If a command descriptor is not available, then the command won't be issued.
Created attachment 91390 [details] patch bringing dstore back from version 9 to 8 In this patch, I'm moved the dstore version back from 9 to 8. In ClientConnection.doHandshake() I've put a check in to consider a client or server version that is mismatched by 1 to still be considered compatible. Any concerns about this approach?
Again, I'd just like to warn about potential maintenance nightmare with being too sloppy on version checks. In my understanding, here's the effects of the current patch: * openRSE client as well as server think they are dstore-8.0.0 * openRSE-3.0 client can handshake with dstore-7.0.0 upto dstore-9.9.9 --> which would include older IBM RSE server --> includes openRSE-1.0.x, 2.0.x and 3.0.x * IBM-RSE client can handshake with openRSE-3.0.x or 1.0.x server (?) * openRSE-2.0.x can NOT handshake with openRSE-3.0.x (because it allowed dstore-9.0.0 only) Now what happens if openRSE client connects to IBM RSE server? * Miner ID's do not match (expected:org.eclipse....; actual: com.ibm.etools...) * Server behavior differs (e.g. cancel token not answered by IBM server) * Daemon port differs I really don't understand what's the benefit of mixing up the versions here. IBM-RSE server and openRSE-server can live peacefully side by side on a host.
(In reply to comment #6) > Again, I'd just like to warn about potential maintenance nightmare with being > too sloppy on version checks. In my understanding, here's the effects of the > current patch: > * openRSE client as well as server think they are dstore-8.0.0 The move from 8.0.0 to 9.0.0 was somewhat unnecessary because the underlying dstore protocol didn't change. > * openRSE-3.0 client can handshake with dstore-7.0.0 upto dstore-9.9.9 > --> which would include older IBM RSE server > --> includes openRSE-1.0.x, 2.0.x and 3.0.x > * IBM-RSE client can handshake with openRSE-3.0.x or 1.0.x server (?) > * openRSE-2.0.x can NOT handshake with openRSE-3.0.x (because it allowed > dstore-9.0.0 only) > Now what happens if openRSE client connects to IBM RSE server? > * Miner ID's do not match (expected:org.eclipse....; actual: com.ibm.etools...) open RSE client can work with the IBM RSE server. The reason is that, with the old IBM rseserver, initialization of the miners occured automatically on the server side when the user connected to it. In open RSE, we use the miner IDs to send commands to initialize the miners but these are unneeded when connecting to the IBM RSE server. > * Server behavior differs (e.g. cancel token not answered by IBM server) With the older server (or more precisely, the old miners) we weren't canceling queries, however the user could still cancel a progress monitor for the client side to stop waiting for something (in this case, the server code would continue to completion). The difference now is that with a new server and client, the backend code does actually cancel. Moving from an old IBM client against an old IBM RSE server to a new open RSE client against the same old IBM RSE server would not result in a change in behaviour for cancelation. Similarly, moving from an old IBM client against and old IBM RSE server to using the old IBM client against the new RSE server would not result in a change of behaviour (since no cancels would be sent). > * Daemon port differs For IBM, the choice of daemon port varies depending on the system and product. The product in this case has it's own daemon and it's got it's own port. They will probably be providing their own modified dstore connector service so the default daemon port will probably be determined by them. > I really don't understand what's the benefit of mixing up the versions here. > IBM-RSE server and openRSE-server can live peacefully side by side on a host. Masao could probably answer this better than I.
(In reply to comment #7) > open RSE client can work with the IBM RSE server. So I wonder how it can identify and talk to the file miner? Also, I'm really scared about what happens to all those changes that we've had in the past where some bugzilla comment said "...this change requires an update to the dstore server in order to work..." -- would all those bug fixes be gone with the old server? Do we have a rough idea about what bug fixes are affected? On a very rough and quick search on bugzilla I found these: bug 180994, bug 214251, bug 181145, bug 173518. What happens when a "change modtime" or "set writable" request is made from openRSE and the ibmRSE server cannot handle it? What happens if openRSE queries the userid, groupid?
(In reply to comment #8) > (In reply to comment #7) > > open RSE client can work with the IBM RSE server. > So I wonder how it can identify and talk to the file miner? The client get's it's command descriptors from the schema (on the client) which got populated by the server. The client finds those descriptors via the command id, rather than the miner so it has no problem getting commands to send to the miner. > Also, I'm really scared about what happens to all those changes that we've had > in the past where some bugzilla comment said "...this change requires an update > to the dstore server in order to work..." -- would all those bug fixes be gone > with the old server? As for the old bugs, well, they'll still be there with the old server but then they were there before in the old versions as well. > Do we have a rough idea about what bug fixes are affected? On a very rough and > quick search on bugzilla I found these: bug 180994, bug 214251, bug 181145, bug > 173518. > What happens when a "change modtime" or "set writable" request is made from > openRSE and the ibmRSE server cannot handle it? What happens if openRSE queries > the userid, groupid?
> I really don't understand what's the benefit of mixing up the versions here. > IBM-RSE server and openRSE-server can live peacefully side by side on a host. Comment from a product architect: While technically it's true that the old and new servers can co-exist, I really don't think that is a very nice solution for the customers. The customers are already complaining about having to configure host servers, and forcing the customers to have to set up 2 host servers and force them to manage who on their team is going to which port, etc is just not the right direction for us to go. Unless there is a strong technical reason why this cannot be done, I really think we should not break support for old clients by new servers and vice versa.
Created attachment 91685 [details] patch providing hooks for compatibility This is an alternative patch which may make more sense going forward. Rather than having ClientConnection do the handshake checking, I've created a new interface, IDataStoreCompatibilityHandler, to take care of that. A product like RDz can provide their own implementation in order to determine whether what level of server it will consider compatible. Any thoughts on this approach?
(In reply to comment #11) > Created an attachment (id=91685) [details] > patch providing hooks for compatibility > This is an alternative patch which may make more sense going forward. Rather > than having ClientConnection do the handshake checking, I've created a new > interface, IDataStoreCompatibilityHandler, to take care of that. A product > like RDz can provide their own implementation in order to determine whether > what level of server it will consider compatible. > Any thoughts on this approach? Note that this approach only deals with the open RSE client. It still doesn't address the case where an IBM client is used against an open RSE server.
(In reply to comment #12) > Note that this approach only deals with the open RSE client. It still doesn't > address the case where an IBM client is used against an open RSE server. Thanks for this extra comment. I think it's important that the RDz team (or others interested in this request) think about whether they want to use the IBM server or the openRSE server in the end. Personally, I think it would make more sense to use the new server because 1.) It's being actively maintained, so if it turns out that it has problems serving the old-style client it can be changed 2.) It's been extended so it's more likely to serve new-style clients well but still be compatible with old-style clients What about going forward with the IDStoreCompatibilityHandler idea, but allowing to register a compatibility handler in the dstore server? - And extending the handshake protocol such that a server can respond with not only one protocol that it supports, but multiple protocols if a compatibility handler is installed (e.g. "Hi. I'm your dstore server. I can talk dstore-7.0, dstore-8.0 or dstore-9.0. What's your preferred protocol?" - Based on the client's answer, the right compatibility mode would then be enabled in the server.
And, having the dstore server support multiple compatible protocols will also benefit the whole open source project in the end (not only the current situation with RDz), because future dstore servers (talking about openRSE 4.0, 5.0,...) would be able to support earlier openRSE clients if needed.
(In reply to comment #13) > What about going forward with the IDStoreCompatibilityHandler idea, but > allowing to register a compatibility handler in the dstore server? - And > extending the handshake protocol such that a server can respond with not only > one protocol that it supports, but multiple protocols if a compatibility > handler is installed (e.g. Currently, the dstore server really doesn't do anything for it's handshake - it simply passes back it's version to the client, and leaves it up to the client to decide whether this is okay or not. I'm thinking one way to deal with the old client connecting to new server issue would be to allow the server version to be specified (overridden) by a JVM property.
Created attachment 91899 [details] patch for setting the server version via property With this patch, you can start the server passing in the following JVM property: -DDSTORE_VERSION=DataStore.<version>.<major>.<miner> For example: -DDSTORE_VERSION=DataStore.8.0.0 This allows you to configure a new server to work with an older RSE client. Will this, along with the compatibility hooks for the client side resolve this?
There is no way to change the JVM property according to the connected client, so that it affects all servers. That is, a new server needs to run in old server mode, until all clients are migrated to a new client. Rather, a user migrated to a new client should be able to take advantage of a new server.
(In reply to comment #17) > There is no way to change the JVM property according to the connected client, > so that it affects all servers. That is, a new server needs to run in old > server mode, until all clients are migrated to a new client. Rather, a user > migrated to a new client should be able to take advantage of a new server. By allowing a product to set the JVM property for a server, the version of a new server can be set such that old clients will still be able to connect there. For new clients, the compatibility handler would account for whatever version has been specified for the server.
The current patches are now in cvs.
An old client can connect to the new server with the DSTORE_VERSION setting. A new client shows "server is older" message. How about sending the client version to the server, and the server sending the real version, if it is newer than DSTORE_VERSION?
I definitely don't like the "off-by-1-is-ok" approach here. A major version change means an incompatible change and needs to be acted on. Anything else distorts the public understanding of version numbers and leads to confusion. DaveM and I discussed this today and we concluded the following: 1. The actual dstore _protocol_ has never changed so far. What's changed is the server framework (in terms of namespace) as well as the miners. But the miners can be queried for their schema and version number already. DStore_9.0.0 has only appeared in milestone builds so far. We will therefore rev down to DStore_8.0.0 again. 2. Any compatibility handlers can only act on previous known versions, they can never look into the future. Therefore, an overall "+-1 major version increment is good with warning, anything else is bad" is not logical and should be removed. ANY major version increment should mean an incompatible change that needs to be acted upon - or it fails. In order to address this: the dstore server should be able to not only send a single (client) version it supports, but a list of (known client) versions that it is able to support. This should be done in a backward compatible way: what used to be the dstore version number will be the lowest dstore protocol version that the server supports, plus additional versions in an additional field. 3. In terms of version numbers, we need to take "branching" into account: * The core openRSE team will be responsible for plain numerical versions e.g. n.n.n * Product teams should be able to modify dstore servers / clients in a way that only this product knows. These modifications can be compatible with the "plain numerical" version or not. Such version numbers should look like this, e.g. 8.1_rdz.0 in order to say that an rdz-specific change was made which is backward compatible with 8.0.0 and 8.1.0 but would be incompatible with 8.1_zos.0 --> These two versions would need to fall back to the 8.1 "common" protocol and disable any rdz / zos specific features. We think that this approach of numerical / ascii versions for branching should account for any possible wish on versions. Product-specific compatibility handlers could perhaps bre registered to make 8.1_rdz compatible with 8.1_zos if the product teams know that these are meant to be compatible; but such behavior is deprecated. Since we are talking about the DSTORE PROTOCOL version which is not really meant to be changed a lot; and, since product teams are encouraged to return their specific dstore protocol changes back to the Open Source Community rather than keep branching their own stuff forever. So, suggested changes (again): 1.) Revert to DStore_8.0.0 --> will fulfill original wish of this bug 2.) Send List of versions rather than single version --> safe for future 3.) Allow num/ascii combination of versions, and change the "compatible check" algorithm to account for the extended format 4.) As a special case, allow the well-known "7.x" versions in openRSE (and probably any other old versions that we know should work). Comments welcome on this recommendation.
Just checked the History of DataStoreAttributes.java, and we'll need to be careful: * We released openRSE-1.0 as "DataStore.8.0.0" * We released openRSE-2.0 as "DataStore.9.0.0" It's probably not such a good idea to release 3.0 as "DataStore.8.1.0" -- we should rather return a list of versions from the server as "7.1.0; 8.0.0; 9.0.0" which the server is able to support. And keep the client at 9.0 but able to handle 7.1.0; 8.0.0; 9.0.0 servers.
(In reply to comment #21) The suggestion looks good to me. In addition to it, I would ask to keep a client version in the server, so that the server can be modified in a compatible way by changing the behavior based on the client version, as the client is doing now.
Good idea. Suggested handshake is as follows: * Server sends list of supported protocol versions to client * Client picks the version that it can best support * IF the server version is >= 9.0.0 (this new feature added), THEN client sends the picked version back to the server So the server will know on what protocol level the client and the server have agreed.
At this point in the release I don't want to go changing the handshake to have the client pass the server it's version. For now, I'm just going to leave the dstore version at 8.0.0. For old IBM versions of dstore (i.e. 7.*) this will allow for 1-off-version compatibilty. For open RSE version 2.0 dstore, where we had marked the version at 9.0.0, this will also work via the 1-off-version compatibility. Going forward, I would expect the versions to rarely go up since dstore is designed to maintain it's current protocol as it evolves. The IDataStoreCompatibilityHandler for the client and version property remain for the server if some customization is required.
For TM 3.0.1, some hard-coded references to "com.ibm..." have been introduced in order to deal with old-style miners. This code should be revisited and properly handled by a backward compatibility handler.
