Community
Participate
Working Groups
Currently the AspectJ build produces the command line distribution consisting of: aspectjrt.jar aspectjweaver.jar aspectjtools.jar + docs, command line launchers, and everything else For consumption by AJDT, the next step is to pull aspectjrt.jar and aspectjweaver.jar into their corresponding plugins, and take a subset of aspectjtools.jar and put that into the ajde plugin. This process has several drawbacks: - the version of AspectJ in AJDT doesn't match the command line builds - the extra steps above are time consuming - much confusion and complication arises when trying to perform Ant builds both in and out of Eclipse Given that AJDT is the primary distribution vehicle for AspectJ, and the importance of having a consistent command line distribution for builds outside of Eclipse, I propose the following: The output of the AspectJ build process includes three jar files, such as: org.aspectj.runtime_1.5.0.jar org.aspectj.weaver_1.5.0.jar org.aspectj.ajde_1.5.0.jar and these end up in the lib directory of the command line distribution, instead of the previous three jars. There also needs to be a new jar, say tools.jar, such that tools.jar + the ajde jar are equivalent to the old aspectjtools.jar. So the command line distro should continue just as before, with these renamed jars. But the trick is that these jars could also contain manifest.mf files which make them OSGi bundles which can be consumed directly by Eclipse. The AJDT build process could then just pick up these jar files as is from the AspectJ build (they would be versioned by the AspectJ build). In addition to improving the above drawbacks, it would also allow for easier end-user upgrades of the AspectJ compiler used in AJDT.
good thing to do in 1.5.1
or not in 1.5.1 ...
could be something happening in the 1.5.3 timeframe for this...
The obvious solution is to convert the existing AspectJ Java projects into plug-in projects. This Apache Harmony model would still allow AspectJ to be tested and packaged as a standalone compiler while also being delivered as a set of plug-ins to Eclipse. There would be additional benefits: 1. AJDT and other consumers could use AspectJ from CVS HEAD for development and optionally build. This would allow commit/patch from a single workspace and easier source level debugging between AJDT and AspectJ. Also a simple runtime workbench can be used for tricky issues such as incremental compilation. 2. OSGi bundle manifests offer a more sophisticated approach to dependency management. For example “Require-Bundle” can be used between the main deliverables i.e. ajdd > tools > weaver > runtime and for separate tests and there respective runtime bundles. Optional dependencies such as XML and JRockit (need to build & test but not shipped) can use “Import-Package”. JDK-specific dependences e.g. aspectj5rt can be implemented as bundle fragments. 3. Improved support for running individual comiler tests especially LTW testing. Currently the CLASSPATH is wrong we tests are run from “tests” as opposed to “run-all-junit-tests” because of missing Java 5 projects e.g. weaver5. The use of “Require-Bundle” and bundle fragments removes this problem.
Should solve Bug 111788 "[build] Unwanted additional files in aspectjtools.jar"
Bug 111788 should be closed or the errant files specified. As I understand it, the org.aspectj module "eclipse.plugins" is responsible for adapting AspectJ builds for AJDT. It's true that this requires Ant now; I wonder if it can be made to run the Ant builder automatically - which means the build module should also produce the required artifacts automatically (not necessarily the installer package, but the artifacts used by eclipse.plugins). Obviously, in this adapter setup the eclipse-destined output (and perhaps the intermediate command-line jars) could be presented as OSGI bundles. I don't really want to go to a world where AJDT as the perhaps-primary consumer gets to determine all features of the AspectJ build artifact interface. Nor am I convinced that adopting or recasting OSGI semantics actually helps. e.g., can it distinguish included or assumed imported libraries (bcel v. XML)? (Right now that's specified as a modifier on a logical dependency.) I would object strongly to only delivering versioned-named jars because that means everyone in the world has to update their scripts to support the new version. Conversely, repartitioning the jars to be more AJDT compliant could help with command-line world; in particular, I think it would be quite useful to have a weaver-only jar. I'm not sure if the AJDT subset of aspectjtools.jar has a standalone use, though. These are all reactive comments; obviously the change has to be planned more systematically. If we do open the build-structure issue, then all build features/enhancements should be considered, and there should be a non-destructive path, so that during the transition, the old builds work. Some of the old design requirements for the build system: - support non-eclipse environments easily - support eclipse consumers like AJDT - dependency changes are easy to make (i.e., no ant build scripts to edit) - capable of easily specifying fairly arbitrary product assemblies - build product in less than two minutes on an average machine and code in less than one minute - robust, e.g., putting a file in the wrong place doesn't make it show up in the build
Wes, Before I respond to the comments I should emphasize that the driving force behind the proposed changes is the relationship between AJDT and AspectJ; while they are separate projects on Eclipse they are in inseparable: AJDT is needed to develop AspectJ and AspectJ is needed to develop AJDT. For those working at the boundary between the two projects the current setup has become untenable: it makes work on the structure model extremely difficult and solving incremental compilation bugs, which are beginning to have a detrimental affect on the adoption of AspectJ, almost impossible. However this is by no means a case of the tail wagging the dog just a realization that the success of AspectJ depends largely on the success of Eclipse+AJDT and the current setup is a hindrance. Documentation on OSGi can be downloaded from http://osgi.org/. However a deep understanding of the specification is unnecessary given the simplicity of bundle manifests and the excellent support in Eclipse PDE. Here is the manifest for org.aspectj.runtime: Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: AspectJ Runtime Bundle-SymbolicName: org.aspectj.runtime Bundle-Version: 1.5.2 Bundle-Localization: plugin Export-Package: org.aspectj.lang, org.aspectj.lang.reflect, org.aspectj.runtime, org.aspectj.runtime.internal, org.aspectj.runtime.internal.cflowstack, org.aspectj.runtime.reflect Eclipse-ExtensibleAPI: true Bundle-RequiredExecutionEnvironment: CDC-1.0/Foundation-1.0 Unfortunately it is not just a question of changing project names. AJDT (and other projects present and future such as AOSGi and Spring) have a bundle dependency on AspectJ. As far as I know bundles cannot depend on regular Java projects. This means that ajde, weaver and runtime must be bundles. Having mixture of project types makes no sense. However what this forgets is that there other benefits of using OSGi bundle modularity besides helping our Eclipse-based consumers many of which I have already described. The most important is what we ship looks like what we develop i.e. if I make a change in org.aspectj.weaver that’s what is shipped. It also means I can make quick versions of a bundle by exporting the project which is very useful when trying to track down bugs. An incremental, non-destructive transition is what has been proposed and documented. I have already created the OSGi bundles listed previous and they satisfy AJDT dependencies (build & test). The use of linked source folders allows synchronization with HEAD (and compatibility with the existing batch build) with the existing projects and development and test with the new projects (something Helen & I have been doing for about a week). I must admit I do not understand the purpose of the eclipse.plugins project. However if I can have a bundle I would rather have a bundle than a process for creating a bundle. I’m not sure I fully understand the question about BCEL and XML but I think the answer is yes. The org.aspectj.weaver bundle includes our bcel.jar and optionally (only needed for LTW) imports XML. This means it can be built and tested with JSE-1.3 under Eclipse. The issue of backwards compatibility with existing Ant scripts was discussed on the call. One idea is to include a (possible empty) tools.jar with the standalone distribution. This would use “Class-Path:” manifest header to connect to the new OSGi bundles. Could you elaborate on the design requirements listed e.g. - support non-eclipse environments easily: what does this mean? - capable of easily specifying fairly arbitrary product assemblies: do you mean runtime + weaver for deployed environments?
org.aspectj Restructure – Phase 1 Tasks: Convert all Java projects to plug-in projects using bundle meta-data instead of classpath to determine inter-module dependencies and versioning Give projects unique names that are the same as deliverable e.g. org.aspectj.runtime so that they can share a workspace with AJDT, OSGi, AOSGi, Spring, … Change module composition and dependencies to be the same as deliverable Put JUnit testcases in separate e.g. org.aspectj.weaver.tests Put projects that comprise a single module together using separate source folders e.g. org.aspectj.weaver contains asm, bridged, loadtime, util and weaver Group and name projects according to purpose: build, test, product e.g. org.aspectj.build.xerces Create linked source folders from new projects to old ones e.g. org.aspectj.runtime/runtime_src > runtime/src Remove org.eclipse packages from jdtcore-for-aspectj.jar because AJDT will need the latest i.e. 3.2 versions. This work is incomplete because the standalone version needs these dependencies. Should create another bundle org.aspectj.eclipse. Create simplified dependency chain ajde > core > weaver > runtime with plug-ins re-exporting packages Use fragments for Java 5 dependent features e.g. org.aspectj.runtime5 Use optional imports for optional dependencies e.g. BEA JRockit Use Eclipse 3.2 Execution Environment instead of JRE so that each project is built with the lowest supported level e.g. CDC 1.0 for org.aspectj.runtime Goals: AJDT (org.eclipse.ajdt) can use AspectJ (org.aspectj) from HEAD org.aspectj.all.tests/RunTheseBeforeYouCommitTests passes org.eclipse.ajdt.tests/AllCoreTests passes Synchronize from HEAD using old projects Catch undesired dependencies in development e.g. accidental 1.4/1.5 API usage Consistent CLASSPATH i.e. Java 5 dependencies available when running under JDK 1.5 without explicit dependency on Java 5 projects or “magic” in the harness Issues: AJDT requires aspectjrt.jar to create ASPECTJ_LIB (https://bugs.eclipse.org/bugs/show_bug.cgi?id=149580). The workaround is to use the binary installed org.aspectj.runtime bundle for running tests and runtime workbenches. AspectJ 1.5.2 Final is looking for aspectjrt.jar on the classpath even for plug-in dependencies in Eclipse (https://bugs.eclipse.org/bugs/show_bug.cgi?id=112098) which means you can’t build AJDT with the new structure. The workaround is to install an older DEVLOPMENT version of AJDT. The org.aspectj.tests project is built with JSE-1.3 but you cannot run individual LTW tests because of the classpath in AjcTestCase (https://bugs.eclipse.org/bugs/show_bug.cgi?id=117885). The workaround is to use JSE-1.5 which includes XML support. This is not a regression because you cannot do this in the existing tests project.
First pass at the list of bundles: New Old org.aspectj.ajde ajde org.aspectj.ajde.tests org.aspectj.all.tests run-all-unit-tests org.aspectj.ant taskdefs org.aspectj.ant.tests org.aspectj.build build org.aspectj.build.apache lib/commons (could be build.commons) org.aspectj.build.bea lib/ext/jrockit org.aspectj.build.tests org.aspectj.build.xerces lib/ant/lib/xml apis.jar;lib/ant/lib/xercesImpl.jar.jar (could be build.xml) org.aspectj.core org.aspectj.ajdt.core;org.eclipse.jdt.core org.aspectj.core.tests org.aspectj.lib org.aspectj.runtime runtime org.aspectj.runtime.tests org.aspectj.runtime5 aspectj5rt org.aspectj.runtime5.tests org.aspectj.testing testing; testing-client; testing-drivers org.aspectj.testing.tests org.aspectj.testing.util testing-util org.aspectj.testing.util.tests org.aspectj.tests tests org.aspectj.tools.ajbrowser ajbrowser org.aspectj.tools.ajbrowser.tests org.aspectj.tools.ajdoc ajdoc org.aspectj.tools.ajdoc.tests org.aspectj.weaver asm; bridge; loadtime; util; weaver; bcel.jar; asm-aj.jar (need a fragment for ASM) org.aspectj.weaver.tests org.aspectj.weaver5 loadtime5; weaver5 org.aspectj.weaver5.tests Every bundle has a tests bundle containing the JUnit tests that were in test-src for the project. The JRE-specific bundles e.g. org.aspectj.runtime5 are fragments. I would like to drop the lib project as it is confusing: it contains code we ship (bcel.jar) and code we don't (JRockit) but just need to build or test with.
AspectJ & AJDT Conference Call discussions (please add anything I missed): - Possible name change org.aspectj.core > compiler - Maybe keep bridge and util separate - Remove JAVA_HOME/lib/tools.jar classpath reference which breaks on Mac OSX - Need for org.aspectj.tools for backwards compatibility with Ant
Over the next week I propose to share the work in progress with the rest of the project. When complete you will need to follow these steps in the specified order: 1. Take a fresh AspectJ workspace in Eclipse 3.2 2. Install JDK 1.5, JDK 1.3 and CDC 1.0/J2ME Foundation 1.0 using the Eclipse Wiki http://wiki.eclipse.org/index.php/Execution_Environments (or get a copy from me). 3. Associate the CDC-1.0, JSE-1.3 and JSE-1.5 Execution Environments with the appropriate JDKs (to avoid build warnings) using the "Window > Preferences... > Java > Installed JREs > Execution Environments" page. 4. Add an "ASPETCTJ_WORKSPACE" Path Variable that identifies the workspace e.g."C:\workspaces\org.aspectj-Restructure" using the "Window > Preferences... > General > Workspace > Linked Resources" preference page. 5. Check out all the new plug-in projects (exact location to be decided). Don't worry about the "The resource is a duplicate of ..." warnings which are cause by using linked source folders. Run "RunTheseBeforeYouCommitTests" in org.aspectj.all.tests as a "JUnit Test" (no need to use "JUnit Plug-in Test") using JDK 1.5 (it will use 1.3 by default). I will need to move some tests from 1.3 to 1.5 because they currently have hidden dependencies. You should be able to develop and test in the new structure then commit using the old (after a refresh of those projects). The names are a little long and may not be easy to distinguish but they are unique and you can always use "Check Out As...". To work with the new bundle projects (unless changing dependencies or adding new projects): 1. Synchronize the old projects with HEAD 2. Refresh (and build) the new projects 3. Run org.aspectj.all.tests 4. Make changes to code in the new projects 5. When adding a test you will need to use the old project to add or modify data e.g. (because there are no linked folders) 6. Run org.aspectj.all.tests 7. Refresh the old projects 8. Synchronize the old projects with HEAD The basic structure is: org.aspectj.ajde > org.aspectj.core > org.aspectj.weaver > org.aspectj.runtime The org.aspectj.weaver module contains asm, bridge, loadtime, util and weaver. To make things easier each project re-exports its public dependencies. In addition there are fragments to hold code with a Java dependency that are automatically loaded with their host when using Java 5 e.g. runtime5 > runtime. Finally each bundle has an accompany tests bundle while org.aspectj.tests contains the compiler tests. Project dependencies (.classpath) are replaced by bundle dependencies (META-INF/manifest.mf). In addition a bundle must be explicit about which packages other bundles can see i.e. its interface must be explicitly exported. There are 3 basic types of dependency: 1. Require-Bundle: Used when a bundle has a hard dependency on another specific bundle implementation e.g. weaver and runtime or weaver.tests and weaver. 2. Import-Package: Used when a dependency is provided by a different bundle during development than when it is deployed e.g. XML. The dependency can be optional as in the case of JRockit. 3. Bundle-Host: Used for JRE-specific features. The fragment is logically part of the host bundle but is only loaded if its requirements are met. This allows weaver.tests to have a hard dependency on weaver but only when run with Java 5 is the weaver5 bundle loaded. The main advantages of this approach are: 1. The project looks like what we ship and so AJDT can consume AspectJ directly from head (so can the Aspects Equinox Incubator and any other Eclipse/OSGi-based project). 2. Better management of dependencies such as Java 5, XML and JRockit. 3. Fewer build breaks because we actually build and test with the Execution Environment that we claim to support i.e. no accidental JDK 1.4 APIs!
I'm +1 for this idea because I think that this will facilitate AJDT development without hindering other clients too much, and the quality of AJDT is a key limiting factor to adoption. Also, in my opinion OSGi is the best and most promising component model available. Some suggested simplifications egarding the proposed naming, since this is a good chance to reduce the number of top-level projects that only serve the purpose of building and testing: 1) org.aspectj.all.tests: bad name, ideally I think that this would be merged into org.aspectj.tests. If not I suggest org.aspectj.tests.junit, org.aspectj.tests.run, or something meaningful along those lines 2) org.apache.build.*: could this be one project with all the different dependencies as seperate build folders? If this stuff were to be depended on outside of the build then it could be named as a dependencies plugin, e.g. org.aspectj.apache, which contains all the Apache dependencies we need. 3) org.aspectj.core: bad name for what's in there, I suggest putting bridge and util into core, and maybe asm. Then call this project org.aspectj.compiler 4) can testing.tests and testing.util.tests be one project that depends on both? 5) can org.aspectj.tests be a testdata type folder inside of testing, then rename testing* to tests in order to follow Eclipse convention?
Mik, 1. The name could be changed but the bundle should not be merged into org.aspectj.tests because that would create unnecessary dependencies i.e. the compiler tests should not depend on individual bundle unit tests. Also org.aspectj.tests.junit is misleading becauses bundles like org.aspectj.runtime.tests _only_ contain JUnit tests. 2. The org.aspectj.build bundles could be named after the provider i.e. merge xerces into apache but keep bea. However I don’t want a single “lib” style bucket. The “build” prefix is needed to distinguish between code need to build the project and code we actually ship hence bcel.jar and asm-aj.jar move to org.aspectj.weaver. 3. I agree and org.aspectj.compiler has been suggested. I don’t agree with moving asm and bridge. If you look at the existing dependencies “bridge” isn’t really a bridge and a separate bundle containing asm , bridge and util would end up below weaver: ajde org.aspectj.ajdt.core loadtime weaver asm bridge util runtime 4. org.aspectj.testing.util contains a single class. It could be consolidated in testing but I had some dependencies problems on my first restructure pass. I will revisit this. 5. No. org.aspectj.testing is the “harness” and org.aspectj.tests contains the compiler tests i.e. JUnit, XML and tests. The former changes rarely while the latter changes all of the time. This is an important separation.
Here is a quick summary of why I think we should switch from Java to plug-in projects. No reason on its own is compelling enough to justify the change but taken in combination I can see no alternative solution. 1. AJDT and other Eclipse/OSGi consumers. Most of our bugs are opened by AJDT users and diagnosing/solving them and providing a solution as quickly as possible in this environment is a priority. 2. Classpath: Currently this is not correct, especially WRT Java 5 dependencies, making it difficult to write certain tests especially LTW. 3. Dependencies: Supporting multiple environments (Eclipse/Ant/runtime, CDC1.0/JSE-1.3/JSE-1.4/JSE1.5/JSE1.6) out of a single deliverable and testing it in a representative way using a development environment (to avoid build breaks) is hard and not achieved completely using the current setup.
I think that the key discussion here is the switch to plug-ins. Regarding the secondary point of structure, I appreciate the points in comment#13 and realize that this is why we have evolved into their current state. However, it is complicated and the structure of the project gets burried in the naming, which results in a very high bar to entry for anyone wanting to contribute or extend. If there isn't a good way to simlify this without trading off too much of the componentization, perhaps creating the multiple roots as others eclipse.org projects have should be considered, e.g.: org.aspectj + plugins + build + tests This is not my favorite approach because it makes it a bit harder to check out a full development workspace, but it could be helpful to those interest in only what's under plugins (for building against and experimentation), people writing patches will additionally have to get tests, and only a few would bother with build.
Mik, I would not be averse to such a structure but you would need another root, dependencies e.g. XML, JRockit, that would always be needed to actually compile the bundles as opposed to packaging them into a standalone deliverable. The tests root would have to contain both testcases and the harness itself i.e. tests vs. testing.
There have been a few issues raised in this discussion which I will try to summarize: 1. Backwards compatibility: One of the aims of any restructure would be to deliver versioned modules that are portable between development (Eclipse), build (Ant) and runtime e.g. org.aspectj.weaver_1.5.3.jar. However we shouldn’t make life hard for owners of existing Ant scripts. One would hope that AspectJ references would be indirected through variables because they already change from release to release e.g. “/aspectj1.2/lib/aspectjrt.jar” to “aspectj1.5/lib/aspectjrt.jar” but we could try to hide any new dependencies e.g. org.aspectj.compiler_1.5.3.jar behind a single tools.jar. 2. Migration: If we do restructure the project we need to do it in stages while continuing to delivering the compiler. I have proposed using linked source folders to allow us to use bundles with the existing batch build. 3. Build system requirements: I may have overlooked or been unaware of some of the goals of the existing project structure and build process. I need to understand these better. 4. Benefits: Other than benefiting AJDT does the use of OSGi bundles help the AspectJ project in other ways? Does it make things more difficult (apart from the obvious impact on build)? Does it give us new opportunities? 5. Alternatives: Could we support the AJDT project without becoming an Eclipse plug-in project ourselves? Would another approach address the issues of managing dependencies (JDK 1.5/1.6) and testing LTW.
Wes, >Nor am >I convinced that adopting or recasting OSGI semantics actually helps. >e.g.,can it distinguish included or assumed imported libraries (bcel >v. >XML)? Right now that's specified as a modifier on a logical dependency.) Does this refer to “skip.libraries” in Builder.properties? In Bug 150487 “Tracing and Logging Framework” I added a dependency on commons.jar which had the side effect of including in the deliverable. I assume that if I include commons.jar in this list, along with similar dependencies like JUnit and Xerces the build will success but the implementation will not be included?
I have just opened Bug 152981 and Bug 152982 to cover the work needed to get Eclipse plug-in projects working.
I believe that AJDT should allow project to use their own aspectjrt.jar. That will preserve consistency with external builds (and, maven, etc). On the other hand you could provide either classpath container or preconfigured user library with aspectjrt.jar packaged with AJDT.
Eugene, Have a look at bug #150494 "Make it possible to choose (external) version of AspectJ to use with AJDT". I filed it earlier this summer, but it haven't received much attention. If you feel that it is somewhat what you mentioned in comment #20, then feel free to add a comment and/or vote for it.
(In reply to comment #21) > Have a look at bug #150494 "Make it possible to choose (external) version of > AspectJ to use with AJDT". I filed it earlier this summer, but it haven't > received much attention. If you feel that it is somewhat what you mentioned in > comment #20, then feel free to add a comment and/or vote for it. I don't know what you mean by "external". AJDT is not using "internal" version, but it does have restrictions on what version can be used. I believe that those restrictions should be removed. That is all.
I meant that, I should be able to point out the exact jar file for AJDT to use. That could e.g. be the jar file that lies in the local repository on my machine. I want to be 100% sure that I have the same build and runtime environment within Ecllipse as outside of it (using e.g. Maven or Ant).
