Community
Participate
Working Groups
See, http://wiki.eclipse.org/EclipseLink/DesignDocs/362899 There are several "NoSQL" databases that are non-relational. These databases include: * Oracle NoSQL * MonogoDB * Hadoop * Google BigTable * Cassandra These databases offer alternative APIs to JDBC and either do not support querying, or support alternative query languages to SQL. You could also include legacy hierarchical and non-relational databases, object databases and XML databases in the "NoSQL" category. It is desirable for EclipseLink to support persistence and specifically JPA access to NoSQL databases. EclipseLink currently provides persistence support for relational databases through JDBC, and non-relational datasources through JCA. This support requires a JDBC or JCA resource adapter/driver for the database. Most NoSQL databases do not provide a standard driver, although some may, and some do provide JDBC drivers. There are also third parties that provide JDBC and JCA driver to a wide range of datasources, including some NoSQL databases. See: * Mongo JDBC Driver * DataDirect * Attunity JCA JDBC Users connecting EclipseLink to NoSQL databases have several options: * If they have a JDBC driver available they can connect using EclipseLink's relational support, creating their own DatabasePlatform in EclipseLink will provide enhanced functionality and ease of use. * If they have a JCA driver available they can connect using EclipseLink's EIS support, creating their own EISPlatform in EclipseLink will provide enhanced functionality and ease of use. * They can implement their own JCA (or JDBC) adapter for their database to enable usage of EclipseLink's EIS support. EclipseLink currently provides several JCA adapters to non-relational datasources. These include: * JMS * Oracle AQ * XML file adapter EclipseLink should provide a platform and adapter to popular NoSQL databases, similar to the platforms provided for popular relational databases. EclipseLink's EIS support currently has several limitations that must be resolved to support NoSQL databases. * No JPA annotation or XML support is currently available for EIS descriptors, mappings, and queries. * Although hooks exists, no current EIS adapters provide Expression (JPQL/Criteria) translation into non-SQL query languages or APIs. * The JPA API has not been tested with EIS descriptors and datasources.
Created attachment 208265 [details] jpa allow eis, logging improvements, pluggable jpql parser
SVN trunk commit: Bug#366426 This patch fixes removes some unnecessary RelationalDescriptor dependencies in JPA, improves logging categorization, and makes the JPQL parser more pluggable. https://bugs.eclipse.org/bugs/show_bug.cgi?id=366426 Changes: - Changed some calls to getPlatform() in JPA, core, testing to getDatasourcePlatform() to support non-relational platforms. - Moved Oracle AQ test resources to Oracle test component. - Fixed Hermes parser to build subselects to avoid duplicate table joins. - Added JPAQueryBuilder/Manager to allow plugable JPQL parsers. - HermesParser: implemented JPAQueryBuilder to allow usage with EclipseLink. - Changed some references to DirectToFieldMapping to AbstractDirectMapping. - Fix Struct to allow be applied to classes only, not fields. - Moved weaving specific property to ClassDescriptor. - Changed JPQL parser usage to use JPAQueryBuilder to allow pluggable parsers. - Added some @Overrides - Made creation of basic/embedded mappings allow descriptor to create correct mapping type. - Fixed EISDirectMapping to account for using correct xpath syntax in field names. - Optimized expression/jpql object IN comparisons to avoid join. - Added support for using isNull/notNull on expression builders/root objects. - Fixed usages of logging to pass log category and use session log when session is available. - Fixed XMLRecord to use XMLField on get(String)/put, not DatabaseField. - Updated the clustering performance tests.
Created attachment 209391 [details] adds support for Oracle NoSQL and MongoDB, adds new eis/eis.test components
Created attachment 209587 [details] adds support for Oracle NoSQL and MongoDB, adds new nosql/nosql.test components
SVN trunk commit: Bug#366426 - NoSQL database support (Oracle NoSQL, MongoDB) This transaction adds two new components to EclipseLink foundation: - nosql - nosql.test These contain support for JPA/ORM access to MongoDB through our EIS support. These components are currently handled by the build the same as our oracle component, in that the main build does not build them, just repackages them into the eclipselink.jar. Also support for Oracle NoSQL database was added to our existing oracle component. These introduce two new dependencies currently not checked in. These jars can be downloaded from Oracle and Mongo, http://www.oracle.com/technetwork/database/nosqldb/downloads/index.html https://github.com/mongodb/mongo-java-driver/downloads or for Oracle committers can be found here, \\otttlfs1\software\NoSQL The MongoDB client jars is already in Orbit, although an earlier version (which is probably compatible and will be upgraded at some point). The build for these new components is still under development, but should be working, and the main build should not be affected. There are two new Eclipse projects for the components and a manifest file, but these are just initial ones, and are probably not completely correct as I am no Eclipse/OSGi expert. These will probably need to be updated. Changes: - antbuild.xml – added optional nosql build to the main ant build - removed crud queries to xml-adapter order tests as now defaulted by the platform - added NoSQL test model - fixed setTableName to set the primary table’s name instead of throw an error if already defined - added nosql.test component to foundation, ant build file, eclipse project - added mongo JPA tests - added Oracle NoSQL dependencies to Oracle build/project - added Oracle NoSQL JPA and native API tests - added DataFormatType enum for NoSql annotation support - added NoSql, Field and JoinField annotations for NoSql support - added nosql.connection-spec, nosql.property, jdbc.connector, jdbc.property to specify driver level properties and connection policies - added default mapping creation to descriptors to create correct mapping type for descriptor type - moved initialize of query manager to descriptor to allow different descriptor types to default different queries - added support for defaulting queries to xml-file adapter - added support to EISDescriptor to fix up XML fields to use the correct XPath - added API on EISPlatform to allow interaction generation for Expressions - fixed some EIS processing of mapped/indexed/xml records - added List APIs to Expression some/all and deprecated Vector APIs - added option to platform to disable prepare of queries - fixed DatabaseField to compare on original field name in addition to uppercase field name, otherwise is not equal to itself and lookups will fail - moved like to regex code to Helper - made CallQueryMechanism compatible with DatasourceCall - fixed class loader usage for JPQL parser factory - added nosql component to foundation, ant build file, eclipse project and manifest - added MongoDBPlatform - added MongoDB JCA adapter - added OracleNoSQLPlatform - added Oracle NoSQL JCA adapter - added unwrap support to EntityManagerImpl for JCA Connection - added support for new nosql and jdbc properties to EntityManagerSetupImpl - added nosql annotation metadata support, but no orm XML support yet
I'm trying to modify my test case to this (using Gemini JPA). Do I have a wrong property when I get this one: Caused by: java.lang.ClassCastException: org.eclipse.persistence.sessions.JNDIConnector cannot be cast to org.eclipse.persistence.eis.EISConnectionSpec at org.eclipse.persistence.eis.EISLogin.getConnectionSpec(EISLogin.java:119) at org.eclipse.persistence.eis.EISLogin.connectToDatasource(EISLogin.java:73) at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.connectInternal(DatasourceAccessor.java:330) at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.connect(DatasourceAccessor.java:418) at org.eclipse.persistence.sessions.server.ConnectionPool.buildConnection(ConnectionPool.java:216) at org.eclipse.persistence.sessions.server.ExternalConnectionPool.startUp(ExternalConnectionPool.java:146) at org.eclipse.persistence.sessions.server.ServerSession.connect(ServerSession.java:484) at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.login(DatabaseSessionImpl.java:652) at org.eclipse.persistence.internal.jpa.EntityManagerFactoryProvider.login(EntityManagerFactoryProvider.java:208) at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:491) ... 30 more
FWIW, I got it working with Gemini JPA with only a few modifications. Currently, Gemini JPA assumes that it must always acquire a data source. Patch attached to bug 369029.
Created attachment 209911 [details] add support for externally managed MongoDb connectivity James, the attached patch adds support for externally managed MongoDb connectivity. This is similar to EclipseLink support for passing a DataSource object on EMF creation. It allows to have an externally managed form of MongoDB connectivity (instead of the persistence.xml). Please let me know what you think about this approach. I'm willing to rework the patch if you prefer a different way.
Created attachment 209916 [details] adds orm.xml for no-sql, and Oracle no-sql enhancements
Created attachment 209917 [details] adds orm.xml support for NoSQL, Oracle NoSQL improvements
SVN trunk commit: Bug#366426, orm.xml support for NoSQL, NoSQL improvements https://bugs.eclipse.org/bugs/show_bug.cgi?id=366426 Code review: Andrei, Guy (pending) Changes: - Added mongo orm.xml tests. - Updated eclipselin-orm schema for NoSQL. - Updated JPA meta-data classes for XML, update mapping project. - Added support for Oracle NoSQL ITERATOR (findAll), putIfAbsent, putIfPresent, putIfVersion, durability, timeouts, consistencies - Changed FieldLockingPolicy to use List/generics/overrides - Add POF config for clustered performance tests - Added Node/NodeImpl classes to expose an asm issue.
The external patch seems fine, I will try to apply it this week or next. Where does your connections come from? EISConnectionSpec does support getting a connections from JNDI, but it would be a JCA Connection. Did you encounter any other issues with Mongo?
(In reply to comment #12) > The external patch seems fine, I will try to apply it this week or next. > Where does your connections come from? EISConnectionSpec does support getting > a connections from JNDI, but it would be a JCA Connection. I'm creating the EMF programmatically in an OSGi environment. The "data sources" (aka. "pools") are configured separately by an administrator at runtime. There may be different databases or pools per tenant. Thus, when creating the EMF I simply set a data source and EclipseLink uses it assuming an externally managed connection pool. I implemented a similar "pool" configuration for Mongo. Thus, at the time my code creates the EMF it already knows about the Mongo pool to use. > Did you encounter any other issues with Mongo? Not yet, I just got to the point where I'm persisting entities. I haven't had a chance to play with queries and advanced operations yet.
Created attachment 210006 [details] adds more mongo functionality, oid, version, sort, select, also UUID support
SVN trunk commit: Bug#366426 - NoSQL support https://bugs.eclipse.org/bugs/show_bug.cgi?id=366426 Code review: Andrei, Guy Changes: - Adds tests for generated value, version locking, order by and select to mongo tests - Adds UUID tests to Oracle NoSQL tests - Adds JPA support for UuidGenerator, metadata, orm.xml - Fixes locking support in EIS accessor - Emulates field/values usage in MappedRecords - Moves some sequence API up from DatabasePlatform to DataSourcePlatform - Makes ReportQuery selects work with EIS records - Adds UUIDSequence - Adds mongo platform support for sort, select, query options, limit, skip - Fixes bug in JPA metadata sequencing from triggering the default sequence causing the platforms default to be overwritten
(In reply to comment #12) > The external patch seems fine, I will try to apply it this week or next. > Where does your connections come from? EISConnectionSpec does support getting > a connections from JNDI, but it would be a JCA Connection. FYI, I just discovered an issue that makes the patch not working. Basically, all "eclipselink.nosql.property.*" properties need to have String values. Objects aren't supported because the PropertiesHandler performs a hard cast to String down the road. As an alternative approach, am I allowed to created my own 'MongoConnectionSpec' and set it via 'eclipselink.nosql.connection-spec'?
Created attachment 210071 [details] patch to isolate nosql tests The patch is to separate Oracle NoSQL tests from Oracle DB Specific tests - Move FL Oracle NoSQL tests out of OracleTestModel - Move JPA Oracle NoSQL tests out of OracleJPATestSuite - Create <test-oracle-nosql> target in foundation\eclipselink.extension.oracle.test\antbuild.xml - Create <oracle-nosql-test> target in jpa\eclipselink.jpa.test\antbuild.xml
Created attachment 210280 [details] patch adds support for external connection factory and fixes 1-m and collections support
SVN trunk commit: Bug#366426, NoSQL external ConnectionFactory support, fixes to collections Code review: Andrei Changes: - Added test for composite id - Added test for OneToMany, ManyToMany, MapKey - Added test for external ConnectionFactory - Fixed JoinFields to be JoinField not Field - Added JPA persistence unit property "eclipselink.nosql.connection-factory" to use external ConnectionFactory - Fixed EISOneToMany to work with mapped data, fixed to allow generated selection query - Fixed JPA to allow non-String properties - Fixed composite collection/direct collection mappings to use ArrayCollectionMappingHelper to merge/compare correctly. - Fixed JPA metadata processing of EISOneToMany mapping and ContainerMappings
Created attachment 210810 [details] patch adds jpql with element collections and native queries for mongo
Created attachment 211414 [details] patch adds jpql with element collections and native queries for mongo
Is this work completed for 2.4? Status is still set to New
fixed
The Eclipselink project has moved to Github: https://github.com/eclipse-ee4j/eclipselink
