Community
Participate
Working Groups
To scale an application beyond a single database machine some sort of mutliple database access is required. Many databases have clustering and replication support, so that mutliple databases can service the application requests. Data can either be replicated, or partitioned across the databases. Ideally would support: - access to multiple data sources (same database platform/schema on each database) - partitioning data for a class across multiple data sources - round robin load balancing - random load balancing - data replication - failover - range based data partitioning - hash based data partitioning - value based data partitioning - code based data partitioning - class level and session level partitioning - Oracle RAC support - WebLogic Gridlink DataSources and Oracle UCP
Created attachment 182617 [details] partitioning support, round robin load balancing and replication
Created attachment 182624 [details] phase 1, round robin, replication
Initial phase of development attached in patch. Adds partitioning policy and support. Provides round robin load balancing and replication support. No meta-data/config, no range partitioning yet.
Changes: - added batch-fetch example - added partitioned, and isolated partitioned version of UnitOfWork test model, uses "virtual rack" (multiple connection pools to the same database) - added partitioningPolicy to ClassDescriptor - added null check to FetchGroupManager to avoid null-pointer on failed deploy - added PartitionPolicy abstract class, defines getConnections API - added ReplicationPolicy, replicates writes to multiple pools - added pool reference to Accessor, so it knows where it came from - added acquire/release connection logging - added partitioningPolicy to AbstractSession - changed AbstractSession accessor to Collection accessors (and updated references) - changed transaction to work with multiple accessors (2 stage commit) - changed call execution to work with multiple accessors - made client sessions (isolated, exclusive) execute calls consistently, added support for partitioning - added @Overrides to sessions, some micro - fixed finally connection release in ReferenceMapping - changed DatabaseQuery accessor to Colleciton accessors - changed SessionBroker getAccessor API to use same getAccessor for partitioning, only a single call, pass query - added setURL to DatabaseLogin - changed ClientSession writeAccessor to Map writeAccessors keyed on pool name - changed ClientSession connection to be lazily assigned - changed getAccessor on ClientSession to assign a connection if in a transaction to support backward compatiblity, and internal usage - changed ServerSession call execution to support partitioning - added JPA partitioned model - changed JPA test framework methods to be instances methods and use inherited getPersistenceUnitName define in test avoid common mistakes in non default unit tests - added JPA paritioning test switch using derby "cluster" and round robin and replication, tests only run on Derby as need to create multiple databases Code review: Andrei (pending)
Created attachment 183338 [details] patch 2, includes config
1 - The new policies have been added to deprecated mappings, namely basic-map and basic-collection (deprecated since 2.0). Wondering if we should not do this? Users should be forced to use element collection if they want to take advantage of the new partition policies? 2 - The new policies can be specified at the @Entity level. What about @MappedSuperclass, @Embeddable? 3 - There seems to be a reference to global partitioning policies, that is, stored on MetadataProject. I didn't see how the policies get set there and re-used by individual classes and/or mappings? Should the policies be supported at the entity-mappings or persistence-unit-defaults level? 4 - I did not see any updates to the OX mapping project XMLEntityMappingsMappingProject for the new schema elements. 5 - The new policies should be added to the merge method from MetadataAccessor. 6 - The last point actually raises a couple other issues. Coding individual metadata objects for each policy currently does not work with the existing xml merging/override. The merge will turn <choice> elements into a multiple selection and in turn, the policy that is applied is in-deterministic to the user. Examples: A) orm.xml - maps Entity A with <round-robin-policy> othermappingfile.xml - maps Entity A with <replication-policy> - This should result in an error. What will happen is that both policies will be merged on Entity A's class accessor and based on the code, the round robin policy will be processed while the replication-policy is ignored. B) orm.xml - maps Entity A with <round-robin-policy> eclipselink-orm.xml - maps Entity A with <replication-policy> - You will get the same result as scenario A). However this is incorrect as the replication-policy should be applied since eclipselink-orm.xml should override. Using an individual PolicyMetadata that accepts a 'type' similar to our <optimistic-locking> would solve this and we would only ever merge a single object. The drawback though, the xml complex type and annotation will contain the metadata for all policy types. The type selected therefore determines the policy metadata that is processed. Also, when using annotations, java does not restrict the user from specifying multiple policies. Therefore, without a validation check during processing, the policy that is applied is in-deterministic to the user. 7 - Noticed a couple TODO's on some of the new policies.
1 - I just added them to all mappings, didn't check if they were deprecated. I will remove them from these. 2 - I think where I put the code covers @MappedSuperclass? Not relevant for Embeddable. 3 - Ya, I have not made them named global elements yet. I started this, but didn't get very far yet, will do that next. 4 - Hmm, seems I missed this. 5 - Ok. When is this used? 6 - I think I was logging a warning if there were multiple policies. Should be throwing an error, but I will eventually allow multiple named policies on a single class. Will try to understand merging better. 7 - I will look into these.
Created attachment 184047 [details] partitioning, named policies, annotations, xml
Hey James, I have a couple of schema suggestions for you: 1. PinnedPartitioning 'connectionPool' has no default in java, but 'connection-pool' is not set to required in the schema. 2. ValuePartitioning 'defaultConnectionPool' has no default in java, but 'default-connection-pool is not set to required in the schema. 3.ValuePartition 'value' has a default while in the schema 'value' is required. 4. some of the partitioning complex types have the 'name' attribute first in the list of attributes and others have it last. Though this doesn't affect the resulting xml file, I think it should be consistent in the schema.
Thanks Karen, I will look into these.
Created attachment 184719 [details] some minor fixes/cleanup
Created attachment 187126 [details] adds data affinity callback support, failover and some performance optimizations
Close for 2.2 release. Will open another bug for additional work. Ignore the last patch, is not in 2.2.
The Eclipselink project has moved to Github: https://github.com/eclipse-ee4j/eclipselink
