Bug 286744 - [Contribution] Add IPC support to ECF
Summary: [Contribution] Add IPC support to ECF
Status: RESOLVED WONTFIX
Alias: None
Product: ECF
Classification: RT
Component: ecf.core (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Chris Aniszczyk CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on:
Blocks:
 
Reported: 2009-08-16 17:58 EDT by Clark Hobbie CLA
Modified: 2014-03-04 16:19 EST (History)
6 users (show)

See Also:


Attachments
CLIPC source code (76.59 KB, application/octet-stream)
2009-08-16 17:58 EDT, Clark Hobbie CLA
no flags Details
Revised source code. (79.77 KB, application/zip)
2009-09-07 17:52 EDT, Clark Hobbie CLA
no flags Details
Proposal #1 for the ECF IPC CVS Layout (264.30 KB, image/jpeg)
2009-09-09 04:32 EDT, Dann Martens CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Clark Hobbie CLA 2009-08-16 17:58:52 EDT
Created attachment 144633 [details]
CLIPC source code

Build ID: I20090611-1540

Steps To Reproduce:
This bug represents the contribution of the CLIPC source code to the Eclipse Communications Framework.

More information:
Please see http://wiki.eclipse.org/IPC_clipc_contribution for more information.

CLIPC is a library containing Java and native elements to facilitate the use of inter-process communications (IPC) in the Java platform.

Please see the above page which contains detailed information regarding the contribution.
Comment 1 Scott Lewis CLA 2009-08-17 18:53:25 EDT
Hi Clark.  Thanks very much for the patch/contribution.  

I'll assign the bug to myself, but just so you know I'm going to have to spread out the necessary work (e.g. review, IP/CQ submission, etc) due to some other requirements on my time.  

I'll try to be as communicative as I can, but if you don't hear from me in timely manner please contact me via this bug, or my email address at slewis at eclipsesource.com.  We'll try to get this pushed through as quickly as possible.

Again, thanks (and inadvance) for your contribution.
Comment 2 Chris Aniszczyk CLA 2009-08-18 23:12:45 EDT
Color me impressed with the thorough contribution, thanks Clark!

I created a CQ for the contribution:

https://dev.eclipse.org/ipzilla/show_bug.cgi?id=3500

Clark, to get this contribution moving, this is what I need you to do:

1) Assert this in this bug:

a. They wrote all the code themselves
b. They have the right to contribute this code
c. They agree with distributing this code under the EPL (every source file
needs an EPL header)

For example,

I am XXX and I assert that:

a.  I authored 100% of the content that I contributed
b.  I have the rights to donate the content to Eclipse
c.  I submit the content under the EPL

2) Refactor the code to the ECF namespace and add the EPL header

I recommend the 'org.eclipse.ecf.ipc' namespace unless anyone has better ideas.

Since you're changing the license to EPL, this is the header that should be in each of your source files:

/*******************************************************************************
 * Copyright (c) 2009 Clark Hobbie and others.
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the Eclipse Public License v1.0
 * which accompanies this distribution, and is available at
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * Contributors:
 *     Clark Hobbie - initial API and implementation
 *******************************************************************************/

Once this is done, please attach the updated source so I can add it to the CQ and get the IP team to start the approval process. It should go quickly since you're the only author :)

3) I believe it makes sense for you to become a committer on the ECF project to maintain this particular contribution. Is that your intention? If so, I will start the committer nomination for you (if Scott approves). I think you would be a great addition to the team!

Let me know and thanks for your contribution.
Comment 3 Chris Aniszczyk CLA 2009-08-18 23:15:42 EDT
I'll look over this contribution.
Comment 4 Chris Aniszczyk CLA 2009-08-18 23:18:29 EDT
Oh, once the refactoring is done and approved.

The ECF team can help with a code review before we do a release ;)
Comment 5 Clark Hobbie CLA 2009-08-19 12:42:18 EDT
Hi Chris & Scott,

Thanks for letting me help out with the Eclipse project.  I've used the Eclipse platform for years and admired the work that you folks have done.

This comment contains the following:
1) Dann Martens
2) Assertion re: authorship, rights, EPL
3) EPL & Header files
4) Package names
5) Committer status


1) Dann Martens

I'd like to include Dann on all the communications because he has been the motive force behind getting CLIPC into ECF.  I'm hoping that Dann and I can continue working together going forward on any modifications that may be needed for the code.

Just to be clear for purposes of the initial contribution, the legal process, etc. The code that I am contributing was written 100% by me --- Dann has been helping with the project stuff, navigating Eclipse contribution process, etc.

2) Assertion re: authorship, rights & EPL

I am Clark N. Hobbie and I assert that

a.  I authored 100% of the content that I contributed
b.  I have the rights to donate the content to Eclipse
c.  I submit the content under the EPL

3) EPL & Header files

The code attached to the bug should already have the EPL header at the top of all the source code so that should be good.  

4) Package names 

I'll rename the base and derived names from "com.lts.ipc" to "org.eclipse.ecf.ipc".  I should have the revised source code soon.

5) Committer status

As you mention it would make sense for me to become a committer on the project and yes that would be my intention.  Two questions about that: a) aside from accepting responsibility for maintaining the code that I am contributing, are there any other responsibilities I need to be aware of?  b) it would make sense for Dann to also be made a committer since he will be working on the project as well.  Could he be made a committer along with me?

I will update the bug with the revised code shortly.

Thanks again,

Clark
Comment 6 Scott Lewis CLA 2009-08-19 13:22:51 EDT
Hi Clark,

First, my apologies...I'm perfectly willing (and wanting) to review this code, comment on it, etc., but I'm in the middle of having to review the work of 3 other google soc projects right now, along with the newsreader contribution...and then there's my full time job...

But I am engaged with this and will work to get things pushed through as quickly as I can.  If Chris and others can help out with any of this then all the better in my view.

(In reply to comment #5)
<stuff deleted>
> 5) Committer status
> 
> As you mention it would make sense for me to become a committer on the project
> and yes that would be my intention.  Two questions about that: a) aside from
> accepting responsibility for maintaining the code that I am contributing, are
> there any other responsibilities I need to be aware of?  

There is a committer agreement with the foundation

http://www.eclipse.org/legal/committer_process/EclipseIndividualCommitterAgreementFinal.pdf

This just outlines responsibilities around/wrt licensing, compliance with EF bylaws, policies, and guidelines.

As for responsibilities WRT ECF as a project...ECF is a little unlike many other Eclipse projects, because the comnmitters are very diverse (i.e. not from a single company).  This means that we/I don't have any formal control over what other ECF committers will do on the project.  I actually like this, because so far it has tended to attract strong, mature, self-motivated committers.  I believe that their commitment to the project comes from their desire to do something interesting, fun, and valuable for the community as well as themselves.  

But it does imply a certain willingness to jump in and contribute...perhaps outside of one's main area of interest on occasion (for example, several ECF committers contribute a *lot* of effort on the ECF build...just to make it possible for us to release with the simultaneous release and get greater distribution for all ECF work).  But just so I'm clear...as project lead I expect people to contribute as much as they can, but can't/won't require anything more than what each committer is willing to give (given other commitments...like jobs/life...of course).

Another thing to keep in mind WRT your contribution...even once a contribution is in place and distributed as part of ECF...that's typically just the beginning...as bug reports, enhancement requests, requests for examples, documentation, presentations, usually appear quickly...and I consider this a good thing.  It means the community is getting value out of something and they would like to get even *more* value out of it.  I expect this will occur with your contribution also.

Sorry for the speach...but I think it's better to be as clear about these things as I can.

b) it would make sense
> for Dann to also be made a committer since he will be working on the project as
> well.  Could he be made a committer along with me?

Yes, certainly.  I hope Dann would/does want to do that.

Thanks.

Comment 7 Chris Aniszczyk CLA 2009-08-19 21:48:14 EDT
I just nominated Clark and Dann to the ECF project. The process takes 7 days for the voting.

Once you post the updated code, I can start the CQ process.

Welcome to the ECF family :)
Comment 8 Clark Hobbie CLA 2009-08-20 09:53:04 EDT
Sounds good.  I'm changing the package name from com.lts to org.eclise.ecf  JNI is a little more finicky about changing package names, so I need to test it out, but I should have it ready by the end of the weekend.

I look forward to working with everyone. :-)

Clark
Comment 9 Chris Aniszczyk CLA 2009-08-31 11:17:21 EDT
Any update on the refactoring :)?
Comment 10 Clark Hobbie CLA 2009-08-31 11:24:23 EDT
During the name change I noticed a bug on the linux platform.  I'm trying to fix that problem before submitting the code.  Sorry about the delay.
Comment 11 Dann Martens CLA 2009-08-31 11:25:43 EDT
(In reply to comment #9)
> Any update on the refactoring :)?

I'm sure it is on its way.

I have two things in the pipeline:
 (1) lining up the build set-up with the other native projects (e.g. Equinox Launcher.
 (2) Creating the bundle/fragment wrappers; as soon as the refactoring is done, I'll update my prototype's and submit everything.
Comment 12 Clark Hobbie CLA 2009-09-07 17:52:40 EDT
Created attachment 146625 [details]
Revised source code.

Version of the source files with the correct package name and a fix for linux semaphores.
Comment 13 Chris Aniszczyk CLA 2009-09-07 19:32:05 EDT
(In reply to comment #12)
> Created an attachment (id=146625) [details]
> Revised source code.

Thanks!

I updated the CQ with the source code. It's going through the approval process now.

I will bug the RT PMC to approve your committership tomorrow. People are on vacation :/
Comment 14 Chris Aniszczyk CLA 2009-09-07 19:33:00 EDT
In the mean time, you may want to work on API docs for your component:

http://wiki.eclipse.org/ECF_API_Docs

Thanks, let me know if you have any questions!
Comment 15 Dann Martens CLA 2009-09-08 03:19:45 EDT
I'll take a look at creating the API docs;
I'll update my bundle prototypes to the contributed code; any suggestions on where these are supposed to go when ready? I can use the equinox native launcher CVS layout as reference.
Comment 16 Chris Aniszczyk CLA 2009-09-08 10:39:55 EDT
Here's a quick question from Tom?

"I would like to know more about the intended use of this library.  Do the
concurrent packages in Java 5 provide much of the same functionality?"

Can you answer it? I believe there isn't full support in Java 5 for what you're providing, correct?
Comment 17 Dann Martens CLA 2009-09-08 10:56:57 EDT
I've done some preliminary work on the API docs, 
 here: http://wiki.eclipse.org/ECF_API_Docs, 
 and here: http://wiki.eclipse.org/ECF_IPC_API_Bundle.

The attached source code seems to be broken as the files supporting the build seem outdated; although that could be related to my set-up. There was a hard-coded MinGW home path, references to legacy 'com.lts.' packages and some variables didn't resolve.

I presume that aligning the build support files with those of e.g. the Equinox Native Launcher is in order now. On a side note, @Chris what part exactly of this contribution are you committing where?
Comment 18 Dann Martens CLA 2009-09-08 11:05:38 EDT
The 'java.util.concurrent' packages in Java are designed to deal with multi-threading and multi-processing. That means they function only inside one-and-the-same virtual machine.

IPC is a set of mechanisms to deal with concurrency issues between different processes running on top of an operating system inside a single host. Granted, the definition of IPC has been expanded historically to include remoting (over different hosts) as well, but this library focuses on local interaction, only.

Java IPC can be used to enable communication between:
 - a Java application running inside a VM and a native process on the same host,
 - two Java applications running inside two different VMs.

There are clear and distinct advantages to IPC, most notably security and performance.
Comment 19 John Arthorne CLA 2009-09-08 11:13:22 EDT
Is there only native code for Linux and Windows? Will the libraries be compiled for 32-bit, 64-bit, or both?
Comment 20 Dann Martens CLA 2009-09-08 11:26:32 EDT
We would like to support all the platforms typically supported by Eclipse.

One of the reasons I've tried (and succeeded) to convince Clark to contribute his work to ECF, is the fact that this community has the know-how and supporting environment to allow his CLIPC library to grow and embrace as much platforms as possible.

I'm using the Equinox Launcher as reference; as time progresses, We hope to have all platforms supported in time for the next release cycle.
Comment 21 Clark Hobbie CLA 2009-09-08 12:03:51 EDT
Re: intended use

java.util.concurrent.Semaphore only works in the same JVM.  If you have two Java programs running in different processes, then java.util.concurrent will not work, CLIPC will.

What's more, if you are trying to synchronize Java and non-Java processes, CLIPC semaphores will work, whereas java.util.concurrent will not.

The basic goals of CLIPC are 1) provide synchronization primitives that simply do not exist in the JRE such as inter-process semaphores and FIFOs; and 2) simplify the primitives that do exist such as shared memory.

Re: build files are broken

Can you be more specific --- is it something in the build.xml or the eclipse project file, etc.

Re: Native code

The code I added only includes code for Linux and Windows.  I have compiled and tested against 32 bit versions.  

There is no reason that I can think of that it would not work on 64 bit versions, but I have not specifically tried those platforms yet.
Comment 22 Dann Martens CLA 2009-09-08 12:24:58 EDT
Hi Clark,

I've used the source package, attached here as 'Revised source code', but couldn't build headless with Ant on Windows. 

There are many possible reasons for this, and quite a few of these reasons are related to my local set-up, but it seems that there are references to non-existing (legacy com.lts.-) packages in the build.xml. A reference to MinGW's home is hard-coded.

Judging from what I've learned so far from the Eclipse Releng Build, some work needs to be done to change the way the IPC libraries are built.

The org.eclipse.ecf.ipc bundle (Java) can be built directly (releng); for each native platform we need to make a fragment available which already contains the compiled native library: releng does not do any native building.

Following the Equinox Launcher example, I've found native build support files somewhere else, but I haven't figured out how theses are related to the releng build support files.
Comment 23 Dann Martens CLA 2009-09-08 12:27:56 EDT
Once the issues surrounding the build support files are resolved, I'll finalize the work already done to compile on (Open)Solaris (x86).
Comment 24 Dann Martens CLA 2009-09-09 04:32:24 EDT
Created attachment 146731 [details]
Proposal #1 for the ECF IPC CVS Layout

After studying the CVS layout and build support files layout of the Equinox Launcher, I would like to propose this layout for the ECF IPC project components.

At the top level, three directories exist: (1) bundles, (2) features and (3) releng.

The bundles directory contains the actual ECF IPC API bundle, and nested inside are the different native fragments for all supported platforms (cfr. launcher bundle). There is also a bundle for tests. The -.library 'bundle' is basically a CDT project which contains all the C code for the individual platforms.

Note that the native fragments contain native artefacts, checking into source control. The Eclipse Releng Build dos *not* build native artefacts. On this level of the build, those native artefacts are mere resources to populate the fragments properly.

The features part is perhaps not necessary in this case, but a possibility nonetheless.

The releng directory contains an Ant build script, to build the native artefacts, specifically from the -.library CDT project. That building happens both locally, as well as remotely. Using SCP and SSH connections are made to the appropriate hardware nodes with the right O/S and compilation is performed, there... 

I still have to figure out when exactly this second-level releng build is kicke d into action and how it relates to the native artefacts already available in CVS. 

Anybody in the circles of Kim Moir who would like to comment, your feedback would be appreciated!
Comment 25 John Arthorne CLA 2009-09-09 14:32:54 EDT
Native artifacts are typically compiled separately outside of the automated build process. This used to be done in a completely manual fashion, but I know the SWT and Equinox launcher people have been experimenting with automation of this step using technologies like Hudson. CCing Andrew who builds the launcher to comment further.
Comment 26 Chris Aniszczyk CLA 2009-09-23 15:14:47 EDT
Clark and Dann, how are things going?

According to the Eclipse Portal, the foundation is just waiting on some paperwork from you to complete the committer process.

Hope all is well!
Comment 27 Dann Martens CLA 2009-09-23 16:38:53 EDT
Hi Chris,

Thanks for the reminder! 

I haven't gotten around to the paperwork yet, for which I apologize.

On the coding side, I'm looking into a way to fit the proposed Java IPC (CLIPC) project structure into the ECF project structure. There are some similarities between ECF and the Equinox project, which I still have to match. Unfortunately, nobody has commented on the structure-related part of my proposal, so far.

For the main Java plug-in and the native fragments (Windows, Linux and Open(Solaris)), I'm happy to attach the code to this bugzilla trail, at this stage. That might help people to play around with the code right now, but I would really like to get confirmation on how to proceed (as in 'consensus').

There will probably be a little push-and-shove until we get it right.   

Best regards,
 Dann
Comment 28 Chris Aniszczyk CLA 2009-10-12 10:29:14 EDT
(In reply to comment #27)
> Hi Chris,
> 
> Thanks for the reminder! 
> 
> I haven't gotten around to the paperwork yet, for which I apologize.

Any updates on the paperwork yet ;)?

We're eagerly awaiting your committership :)
Comment 29 Dann Martens CLA 2009-10-13 08:21:01 EDT
Thanks. I have sent the Committer Agreement to EMO.

In the meanwhile, I'll continue on creating the ECF IPC native libraries for 64-bit Linux platforms.
Comment 30 Chris Aniszczyk CLA 2009-10-21 15:29:42 EDT
(In reply to comment #29)
> Thanks. I have sent the Committer Agreement to EMO.

Thanks, I've confirmed with the EMO that they have your paperwork in order. Unfortunately we have to restart the election since the 60 days passed. booo...

On the brightside, things should go smooth now but will take 7 days from today.

I believe Clark is still required to finish some paperwork according to the EMO.

--

Also, the EMO has some questions about the contribution still:

Hi Chris:

Clark is not yet a committer and so I can't add him to this CQ.

I've triaged the content and have a few questions:

Both Readme files appear to contain information relevant to its existence
outside of Eclipse.  Further it appears they may still be referencing the
copying of the content related to the LGPL license.  Are these files to be
included?

Can you please advise.

--

Thoughts?
Comment 31 Dann Martens CLA 2009-10-21 15:48:11 EDT
From what I read in those files, it appears to be mere background information 
explaining the transition from the Sourceforge to the Eclipse repo's.

I can't find any references to the LGPL license, the copyright headers seem to be in order, and there is an additional epl-v10.html in several misc folders.

I have used Clark's Revised source code attachment as reference.
Comment 32 Clark Hobbie CLA 2009-11-20 10:08:13 EST
Apologies for taking so long with the paperwork, etc.  I just submitted the individual committer agreement as well as the employer stuff to emo@eclipse.org.  

I realize the previous vote expired/timed out.  Assuming that I can get the committer process started again, what do I need to do?
Comment 33 Chris Aniszczyk CLA 2009-11-20 10:21:21 EST
I kickstarted the election Clark just now.

The votes will have to be reissued and we will get the results 7 days from now.

Dann should have commit access to ECF now and he should be able to commit the initial contribution now.
Comment 34 Scott Lewis CLA 2009-11-20 10:47:23 EST
(In reply to comment #33)
> I kickstarted the election Clark just now.
> 
> The votes will have to be reissued and we will get the results 7 days from now.
> 
> Dann should have commit access to ECF now and he should be able to commit the
> initial contribution now.


Please commit projects to the module:

org.eclipse.ecf/incubation/bundles
Comment 35 Scott Lewis CLA 2014-03-04 16:19:14 EST
closing as the IPC contribution has been orphaned.