[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [wtp-dev] The Big Picture

With a co-worker's help, I added transitive reduction to the output graph. The transitive reduction removes direct dependencies between components if the dependency can be satisfied indirectly. For instance, if you have A -> B -> C and also A -> C, the transitive reduction will remove the A -> C. Attached is the graph of the same component definitions as before, but this time after applying transitive reduction (and using a different graph layout). The transitive reduction does remove information from the graph and so is not a substite for the original graph, but it does make certain dependency patterns easier to spot and creates a more readable architecture diagram for documentation purposes.
 
Here is an example of something that caught my eye and may be worth discussing at some point. Notice that the Source Editors component depends on the ModuleCore (the WTP project structure/model) component. Does this mean that it would be impossible for an adopter to re-use our source editors outside the context of WTP projects? Would it be worth it to try to put an abstraction layer/interface between the Source Editors component and ModuleCore?
 
- Konstantin
 


From: wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Konstantin Komissarchik
Sent: Thursday, September 06, 2007 11:46 AM
To: General discussion of project-wide or architectural issues.
Subject: RE: [wtp-dev] The Big Picture

Tim,
 
Thanks for comitting to organize this effort. We can certainly collaborate on the architecture tools and I don't have a problem releasing the source under EPL or sharing otherwise. The one problem is that my utility performs the analysis and then generates a text file that is consumed by Graphviz (http://www.graphviz.org). With the new IP rules now in place, even though I wouldn't be actually re-distributing Graphviz (which is under a CPL license), it could be argued that my tool is not usable without Graphviz, so I am in fact re-distributing it. This basically means, that we would have to run Graphviz through the full IP process before releasing any of my code to a CVS server maintained by the Eclipse Foundation. Sigh. I will run these specifics by Janet to get her reading on this. Doesn't mean we couldn't collaborate in the meantime. We would just need to find some creative ways to do that.
 
Regarding the issue of including the test plugins... As we find cases where test plugins cross component-boundaries, I imagine we would want to create a cross-wtp test component to hold those tests. Overall, most test plugin seem to stay within component boundaries, which makes it easier.
 
In the meantime, since starting this discussion, I have been pushing on improving my tool and componentizing the Common Project. Attached is the current work-in-progress. Don't read too much into this. It is a work-in-progress, was not reviewed by anyone else, will change significantly, etc. I just wanted to share what I've been doing with the rest of the team. This is based on 3.0 M1 with a patch for https://bugs.eclipse.org/bugs/show_bug.cgi?id=202488 that resolves one of the problematic dependencies.
 
components.xml - the component definitions that I feed my utility.
components-and-dependencies.txt - lists the plugins by component (after the regex patterns in component.xml are resolved); shows dependencies of each component on other plugins
big.picture.txt - the graph definition that can be fed to Graphviz
big.picture.png - the rendering of big.picture.txt using the circular graph layout of Graphviz. This particular layout is real good for spotting simple cycles (just look for parallel lines). To make spotting cycles even easier, my tool detects cycles and tags components that are not involved in cycles as green.
 
- Konstantin
 
 


From: wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Tim deBoer
Sent: Thursday, September 06, 2007 9:25 AM
To: General discussion of project-wide or architectural issues.
Subject: RE: [wtp-dev] The Big Picture


Hi Konstantin,

Thanks for building this tool - the graphs really help to provide an overview of the dependencies and should generate some good discussion. I also have a home brew tool that scans for dependencies and provides information on references, calculates install sizes, etc., so maybe we can find a way to collaborate.

> I then ran my utility against 3.0 M1 installation which included WTP SDK and WTP automated tests.

We may want to compare without test plugins as well - I know in some cases teams have built test harnesses that aren't strictly component specific, e.g. a test that extends WTP like an adopter to test extension points and interactions across components. We'll need to examine these dependencies to understand if they are valid, mistakes, or if there is some restructuring that should occur. All of this should be done in light of the installable options we want to provide in addition to our current features and new component definitions.

I have to send my regrets for today's call, but David will be announcing the formation of an Architecture Working Group. I'll setup a call next week so that we can start talking about these issues, update our current docs, and decide what work needs to be done.

Thanks,
Tim deBoer
deboer@xxxxxxxxxx



"Konstantin Komissarchik" <kosta@xxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx

09/04/2007 09:27 PM

Please respond to
"General discussion of project-wide or architectural issues."        <wtp-dev@xxxxxxxxxxx>

To
"General discussion of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
cc
Subject
RE: [wtp-dev] The Big Picture





One of my goal was to better understand the new organization as specified by the project refactoring proposal, so using existing features would not work for that since they have yet to be updated to reflect this. Part of this process (at least as far as it applies to common) is to drive the creation of new features.
 
So... If you look at the component listings in my e-mail, you will notice that WST and JST parts co-mingle (just like they do in the project refactoring proposal). Now, if under each of the new projects we define a WST and a JST component and graph dependencies of those components, then we would not end up with any cycles (the build works, as you point out). Is that good enough? Is it ok to have cycles at project-level as long as we can define components such as no cycles would exists? How would you build such a structure once the codebase is re-organized based on the project refactoring proposal? We wouldn't be able to have a project-level build since a top-level driver would need to know how to weave builds of components of different projects such that cycles are avoided.
 
The Java EE <-> Common cycle is due to jst.common.frameworks depending on wst.web. This seems wrong to me. This builds right now because by the time jst.common builds, all of wst has been compiled, but I think jst.common should only be able to see wst.common.
 
- Konstantin
 
 


From: wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of David M Williams
Sent:
Tuesday, September 04, 2007 5:57 PM
To:
General discussion of project-wide or architectural issues.
Subject:
Re: [wtp-dev] The Big Picture



I'm not sure I understand, but if you are saying that Common depends on Java EE, then I think something is amiss.
The way our build works, it wouldn't compile if that was the case.


Does your analysis also take into account the WST/JST split?


I think the way to approach this would be more along the lines of our defined features?


Or, am I simply not understanding?


thanks,




"Konstantin Komissarchik" <kosta@xxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx

09/04/2007 08:19 PM

Please respond to
"General discussion of project-wide or architectural issues."        <wtp-dev@xxxxxxxxxxx>


To
"General discussion of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
cc
Subject
[wtp-dev] The Big Picture







So in my quest to understand the "Common Project", I've been writing a little utility that can analyze plugin dependencies and produce a component-to-component dependencies (given component definitions). Until today, I've been mostly focused on understanding how the rest of WTP depends on common, but today I decided to let my utility show me the rest of the picture. Attached is the result.

 

I knew the dependency lines were not going to be clean, but I was a bit surprised by the result. There are four simple circles that can be easily spotted...

 

Java EE <-> Common (yikes!)

Java EE <-> EJB

Java EE <-> Server Tools

Java EE <-> Web Services

 

More complicated circles are also present, but these are harder to spot. One example is Web Services -> EJB -> Java EE -> Web Services.

 

It strikes me that we either have a lot of work ahead of us on refactoring the dependencies or these boxes aren't quite right. Maybe a bit of both.

 

Thoughts? Does any of this come as a surprise?

 

I used information found in
http://www.eclipse.org/webtools/devProcess/wtpProjects/Refactor%20WTP%20Proposal.pdf as the definition of the component boundaries. I then ran my utility against 3.0 M1 installation which included WTP SDK and WTP automated tests. Let me know if the component membership as listed is not accurate and needs to be tweaked. The only two plugins that are not accounted for are org.eclipse.wst and org.eclipse.jst. Not sure where these would end up.
 

Common Component


org.eclipse.jem
org.eclipse.jem.beaninfo
org.eclipse.jem.proxy
org.eclipse.jem.tests
org.eclipse.jem.ui
org.eclipse.jem.util
org.eclipse.jem.workbench
org.eclipse.jst.common.annotations.controller
org.eclipse.jst.common.annotations.core
org.eclipse.jst.common.annotations.ui
org.eclipse.jst.common.frameworks
org.eclipse.jst.common.project.facet.core
org.eclipse.jst.validation.sample
org.eclipse.jst.validation.test
org.eclipse.wst.command.env
org.eclipse.wst.command.env.core
org.eclipse.wst.command.env.doc.user
org.eclipse.wst.command.env.infopop
org.eclipse.wst.command.env.ui
org.eclipse.wst.common.core
org.eclipse.wst.common.emf
org.eclipse.wst.common.emfworkbench.integration
org.eclipse.wst.common.environment
org.eclipse.wst.common.frameworks
org.eclipse.wst.common.frameworks.ui
org.eclipse.wst.common.infopop
org.eclipse.wst.common.modulecore
org.eclipse.wst.common.project.facet.core
org.eclipse.wst.common.project.facet.core.tests
org.eclipse.wst.common.project.facet.ui
org.eclipse.wst.common.project.facet.ui.tests
org.eclipse.wst.common.snippets
org.eclipse.wst.common.snippets.tests
org.eclipse.wst.common.tests
org.eclipse.wst.common.tests.collector
org.eclipse.wst.common.tests.ui
org.eclipse.wst.common.ui
org.eclipse.wst.common.ui.properties
org.eclipse.wst.common.uriresolver
org.eclipse.wst.internet.cache
org.eclipse.wst.internet.cache.tests
org.eclipse.wst.validation
org.eclipse.wst.validation.infopop
org.eclipse.wst.validation.ui

 

Server Component


org.eclipse.jst.server.core
org.eclipse.jst.server.core.tests
org.eclipse.jst.server.generic.core
org.eclipse.jst.server.generic.jboss
org.eclipse.jst.server.generic.jonas
org.eclipse.jst.server.generic.oc4j
org.eclipse.jst.server.generic.tests
org.eclipse.jst.server.generic.ui
org.eclipse.jst.server.installable
org.eclipse.jst.server.preview.adapter
org.eclipse.jst.server.tomcat.core
org.eclipse.jst.server.tomcat.core.tests
org.eclipse.jst.server.tomcat.ui
org.eclipse.jst.server.tomcat.ui.tests
org.eclipse.jst.server.ui
org.eclipse.jst.server.ui.doc.user
org.eclipse.jst.server.ui.infopop
org.eclipse.jst.server.ui.tests
org.eclipse.jst.server.websphere.core
org.eclipse.wst.internet.monitor.core
org.eclipse.wst.internet.monitor.ui
org.eclipse.wst.internet.monitor.ui.tests
org.eclipse.wst.server.core
org.eclipse.wst.server.core.tests
org.eclipse.wst.server.http.core
org.eclipse.wst.server.http.core.tests
org.eclipse.wst.server.http.ui
org.eclipse.wst.server.preview
org.eclipse.wst.server.preview.adapter
org.eclipse.wst.server.ui
org.eclipse.wst.server.ui.doc.user
org.eclipse.wst.server.ui.infopop
org.eclipse.wst.server.ui.tests

 

Source Editors Component


org.eclipse.jst.jsp.core
org.eclipse.jst.jsp.core.tests
org.eclipse.jst.jsp.tests.encoding
org.eclipse.jst.jsp.ui
org.eclipse.jst.jsp.ui.infopop
org.eclipse.jst.jsp.ui.tests
org.eclipse.jst.standard.schemas
org.eclipse.wst.css.core
org.eclipse.wst.css.core.tests
org.eclipse.wst.css.tests.encoding
org.eclipse.wst.css.ui
org.eclipse.wst.css.ui.tests
org.eclipse.wst.dtd.core
org.eclipse.wst.dtd.ui
org.eclipse.wst.dtd.ui.infopop
org.eclipse.wst.dtd.ui.tests
org.eclipse.wst.dtdeditor.doc.user
org.eclipse.wst.html.core
org.eclipse.wst.html.core.tests
org.eclipse.wst.html.tests.encoding
org.eclipse.wst.html.ui
org.eclipse.wst.html.ui.infopop
org.eclipse.wst.html.ui.tests
org.eclipse.wst._javascript_.core
org.eclipse.wst._javascript_.ui
org.eclipse.wst._javascript_.ui.infopop
org.eclipse.wst.sse.core
org.eclipse.wst.sse.core.tests
org.eclipse.wst.sse.doc.user
org.eclipse.wst.sse.ui
org.eclipse.wst.sse.ui.infopop
org.eclipse.wst.sse.ui.tests
org.eclipse.wst.standard.schemas
org.eclipse.wst.xml.core
org.eclipse.wst.xml.core.tests
org.eclipse.wst.xml.tests.encoding
org.eclipse.wst.xml.ui
org.eclipse.wst.xml.ui.infopop
org.eclipse.wst.xml.ui.tests
org.eclipse.wst.xml.validation.tests
org.eclipse.wst.xmleditor.doc.user
org.eclipse.wst.xsd.core
org.eclipse.wst.xsd.ui
org.eclipse.wst.xsd.validation.tests
org.eclipse.wst.xsdeditor.doc.user

 

Web Services Component


org.eclipse.jst.ws
org.eclipse.jst.ws.axis.consumption.core
org.eclipse.jst.ws.axis.consumption.core.tests
org.eclipse.jst.ws.axis.consumption.ui
org.eclipse.jst.ws.axis.creation.ui
org.eclipse.jst.ws.axis.infopop
org.eclipse.jst.ws.axis.ui.doc.user
org.eclipse.jst.ws.axis2.consumption.core
org.eclipse.jst.ws.axis2.consumption.ui
org.eclipse.jst.ws.axis2.core
org.eclipse.jst.ws.axis2.creation.core
org.eclipse.jst.ws.axis2.creation.ui
org.eclipse.jst.ws.axis2.ui
org.eclipse.jst.ws.consumption
org.eclipse.jst.ws.consumption.infopop
org.eclipse.jst.ws.consumption.ui
org.eclipse.jst.ws.consumption.ui.doc.user
org.eclipse.jst.ws.creation.ejb.ui
org.eclipse.jst.ws.creation.ui
org.eclipse.jst.ws.doc.user
org.eclipse.jst.ws.infopop
org.eclipse.jst.ws.tests
org.eclipse.jst.ws.uddiregistry
org.eclipse.jst.ws.ui
org.eclipse.wst.ws
org.eclipse.wst.ws.explorer
org.eclipse.wst.ws.infopop
org.eclipse.wst.ws.parser
org.eclipse.wst.ws.tests
org.eclipse.wst.ws.ui
org.eclipse.wst.wsdl
org.eclipse.wst.wsdl.tests
org.eclipse.wst.wsdl.tests.ui
org.eclipse.wst.wsdl.ui
org.eclipse.wst.wsdl.ui.doc.user
org.eclipse.wst.wsdl.validation
org.eclipse.wst.wsdl.validation.tests
org.eclipse.wst.wsi
org.eclipse.wst.wsi.tests
org.eclipse.wst.wsi.ui
org.eclipse.wst.wsi.ui.doc.user

 

Java EE Tools Component


org.eclipse.jst.doc.isv
org.eclipse.jst.j2ee
org.eclipse.jst.j2ee.core
org.eclipse.jst.j2ee.core.tests
org.eclipse.jst.j2ee.doc.user
org.eclipse.jst.j2ee.infopop
org.eclipse.jst.j2ee.jca
org.eclipse.jst.j2ee.jca.ui
org.eclipse.jst.j2ee.navigator.ui
org.eclipse.jst.j2ee.tests
org.eclipse.jst.j2ee.ui
org.eclipse.jst.j2ee.web
org.eclipse.jst.j2ee.webservice
org.eclipse.jst.j2ee.webservice.ui
org.eclipse.jst.j2ee.xdoclet.runtime
org.eclipse.jst.jee
org.eclipse.jst.jee.ui
org.eclipse.jst.jee.web
org.eclipse.jst.servlet.tests
org.eclipse.jst.servlet.ui
org.eclipse.jst.servlet.ui.infopop
org.eclipse.wst.doc.isv
org.eclipse.wst.doc.user
org.eclipse.wst.web
org.eclipse.wst.web.ui
org.eclipse.wst.web.ui.infopop
org.eclipse.wst.webtools.doc.user

 

EJB Tools Component


org.eclipse.jst.ejb.doc.user
org.eclipse.jst.ejb.ui
org.eclipse.jst.j2ee.ejb
org.eclipse.jst.j2ee.ejb.annotation.model
org.eclipse.jst.j2ee.ejb.annotations.emitter
org.eclipse.jst.j2ee.ejb.annotations.ui
org.eclipse.jst.j2ee.ejb.annotations.xdoclet
org.eclipse.jst.jee.ejb

 

JSF Component


org.eclipse.jst.jsf.common
org.eclipse.jst.jsf.common.ui
org.eclipse.jst.jsf.contentassist.tests
org.eclipse.jst.jsf.context.symbol.tests
org.eclipse.jst.jsf.core
org.eclipse.jst.jsf.core.tests
org.eclipse.jst.jsf.designtime.tests
org.eclipse.jst.jsf.doc.dev
org.eclipse.jst.jsf.doc.user
org.eclipse.jst.jsf.facesconfig
org.eclipse.jst.jsf.facesconfig.ui
org.eclipse.jst.jsf.metadata.tests
org.eclipse.jst.jsf.metadataprocessingtests2
org.eclipse.jst.jsf.standard.tagsupport
org.eclipse.jst.jsf.test.util
org.eclipse.jst.jsf.ui
org.eclipse.jst.jsf.ui.tests
org.eclipse.jst.pagedesigner
org.eclipse.jst.pagedesigner.jsf.ui
org.eclipse.jst.pagedesigner.jsp.core

 

JPT Component


org.eclipse.jpt
org.eclipse.jpt.core
org.eclipse.jpt.core.tests
org.eclipse.jpt.core.tests.extension.resource
org.eclipse.jpt.db
org.eclipse.jpt.db.ui
org.eclipse.jpt.doc.user
org.eclipse.jpt.gen
org.eclipse.jpt.ui
org.eclipse.jpt.utility
org.eclipse.jpt.utility.tests

 
 


  Konstantin Komissarchik
Staff Software Engineer
Phone: 206.926.2962
IM: kosta0120 on Yahoo

kosta@xxxxxxx
BEA Systems, Inc.
999 North Northlake Way
Seattle, WA 98103

www.bea.com
  Barring that natural _expression_ of villainy which we all have, the man looked honest enough.
                                                 
 


                                                   Mark Twain              



 

Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev


Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev


Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.

Attachment: big.picture.35.png
Description: big.picture.35.png