Re: [science-iwg] About java-python integration

[Tracy Miranda <tracy@xxxxxxxxxxxxxxxx>]

Hi Michele,

As you've observed, jpy was not a library we were aware of when we did the evaluation of java-python solutions. I'd be interested in any insights you have with regard to its approach. We have settled on Py4J as the java-python bridge which we are settling down with, building a house & having babies... :-)

Here is the criteria we used to evaluate the bridges at the time, it might be useful to you should you chose to do your own evaluation for specific use-cases.

Technical merit:

--------------------

What is overall architecture, who manages the interpreter lifecycle? (e.g. Py4J has separate Python/Java processes, uses sockets)

Does the implementation provide full support for C-libraries or conditional/partial support? (Py4J works with an independent Python process so full numpy support)

Does the library handle memory management (e.g. garbage collection on objects shared between Python and Java processes)?

What is the threading model between Java/Python

Related to threading, Is there support for indirect recursive calls between Java & Python? (Our recent work involved adding this feature to Py4J)

Is remote support possible or on the roadmap?

Does the library pass data and objects by call-by-ref or call-by-value (i.e. copying) or a combination of both.

Is there evidence of a good testsuite for the functionality?

Performance!!!! (this is a whole huge area in itself, so depends what you are optimizing for e.g individual calls vs batch, in due course we would like to formally measure & publish some figures for the solutions we are working on)

Non-technical considerations:

-------------------------------------

What is the project licensing?

Are there any platform support limitations (We need to support Windows, Mac, Linux)

What evidence is there the project is alive and thriving (look for releases in last 6 months, activity on forums)

Look for evidence of adoption (e.g Py4J is used by Apache Spark for PySpark, they also make contributions)

Look for other evidence of sustainability (any funding, companies, institutions involved)

Is the project open to external contributions?

Is there a roadmap?

Is there sufficient documentation?

There was no one perfect solution, but Py4J fulfilled many of the requirements and we are working on filling in the gaps. 

Let me know if any questions, otherwise I will (separately) send out an update on our Py4J activities especially after our productive week at EclipseCon France last week. 

Regards,

Tracy

On Mon, Jun 13, 2016 at 4:17 PM, Michele Gabusi <michele.gabusi@xxxxxx> wrote:

Hi,

thank you all for your answer. They have been very instructive indeed.

jpy is distributed with an Apache License from 0.8 version (on github); however, the previous release - which is available on pip right now - is GPL. Hopefully in future license should not be a serious problems. Thank you for having clarified the other ways to integrate java and python. Actually I was partially aware of them (thanks to the EclipseCon presentations), but now the picture is clearer.

Cheers,

Michele.   

Il 11/06/2016 09:41, Christian Pontesegger ha scritto:

It seems jpy (at least now) is using an Apache 2.0 license, so there would not be any license problems.

Jonah is currently implementing python support using Py4j and we already have a well working prototype. If jpy would have benefits over Py4j it would be worth considering an implementation for that, too.

Christian

On 06/10/2016 05:33 PM, Michele Gabusi wrote:

Hi,

recently I was looking around for a handy solution to setup bidirectional communication between Java and Python. I know that this is not a trivial issue. However, looking at the past EclipseCon presentations, I noticed that jpy libraries (licensed under GPL3) have not been mentioned.

http://jpy.readthedocs.io/en/latest/intro.html
https://pypi.python.org/pypi/jpy/

Do you know any reason that might discourage to implement them?
Do anyone has any experience with these libraries?

Cheers,

Michele.


_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg

_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg

--

Michele Gabusi
Data Mining and Business Analytics
Big Data Competency Center
michele.gabusi@xxxxxx

Engineering Ingegneria Informatica
Corso Stati Uniti, 23/C - 35127 Padova - Italy
Tel. +39-049.8283549
Fax +39-049.8283569
www.eng.it

_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg