Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [wtp-dev] M4 and testing (as M3 and testing)


David,

Your reasoning seems to imply that we aren't changing anything, i.e. existing clients will continue to work. I think we need to allow freedom to change interfaces, and potentially to break existing clients.

I hope that as we look at our current implementation we will see areas that need improvement, e.g. missing methods, incorrectly named methods, etc. Perhaps in some cases we need to create new interfaces altogether. Designers should have the freedom to create a well-crafted API, and they should do this before they fill in the implementation, even if they end up reusing a lot of existing code for the implementation. JUnit test cases should be constructed to express the desired behavior of the API since we need to preserve both the syntax and the semantics of APIs in future releases.

We need to allow adequate time for public review of API and for us to respond to feedback. Therefore it is more important to define the API before implementing it. If APIs have working JUnits, great. But if we need to priortize something, then the implementation of the API is lower priority than the definition of the API. Our exit criteria for M4 should therefore not require all API JUnit tests to pass.

Arthur Ryman,
Rational Desktop Tools Development

phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
intranet: http://labweb.torolab.ibm.com/DRY6/



David M Williams <david_williams@xxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

03/08/2005 12:57 PM

Please respond to
wtp-dev

To
wtp-dev@xxxxxxxxxxx
cc
Subject
Re: [wtp-dev] M4 and testing (as M3 and testing)






There's an possible technical answer to this issue ... and that's just to keep the non-implemented API tests temporarily in a separate plugin that's not included in I-builds and Milestone builds, just nightly builds, where they would fail. But I can't help but think this would be a very rare case. After all, to be API is, in part, to have clients, so to have clients, there must be an implementation. And, its my understanding, an API without an implementation is not really an API yet, as it would surely change some as its implemented and change some as its used by clients.


More importantly, I think there's a lot of value in having the JUnit tests serve as a BVT, "Build Verification Test" ... that is, confirms that the build would be expected to basically run ok, even if not perfectly ... so would hate to set a precedent that its ok to have red X's on build page.


While I fully agree with the principles of "API First", we can't ignore that we already have lots of implementation, and lots of undocumented APIs, and I suspect 80% of our efforts is documenting the APIs we think we have, fine-tuning and finalizing them, not, in other words, coming up with new ones.


I don't mean any of this as disagreement with the goals having platform quality APIs, just to further this discussion as to how to get there.


Perhaps its a more clear concrete goal to state that API are to be defined, documented, and have working JUnit tests by M4. Anything that's not in that category would have to go through the API approval process, as has been suggested on this list by another post from Arthur?


Suggestions/comments welcome.


David



Arthur Ryman <ryman@xxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

02/24/2005 04:09 PM

Please respond to
wtp-dev

To
wtp-dev@xxxxxxxxxxx
cc
Subject
Re: [wtp-dev] M3 and testing








Looking forward to M4, we should be developing API First and being Test-Driven. This means that it is OK for some JUnit tests to fail at M4 exit, i.e. those that test API implementation. In fact, I expect the implementation to only be complete by M5.


We need to identify the subset of JUnit tests that need to pass for M4 and beyond, i.e. not let just any JUnit failure make the build fail and not have to pull test cases from the build.


Arthur Ryman,
Rational Desktop Tools Development

phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
intranet: http://labweb.torolab.ibm.com/DRY6/

Chuck Bridgham <cbridgha@xxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

02/24/2005 08:46 AM

Please respond to
wtp-dev


To
wtp-dev@xxxxxxxxxxx
cc
Subject
Re: [wtp-dev] M3 and testing










Hi Naci,


We have been making incremental improvements to the junit's success, but all problems remaining are setup/junit issues, not blocking functionality.

I would like to spend a little more time investigating these, and if not successful, we will pull these test cases from the build.


I do not think this should delay our declaration of M3.   We will be smoke testing the latest build this morning.


Thanks - Chuck


Rational J2EE Tooling Team Lead
IBM Software Lab - Research Triangle Park, NC
Email:  cbridgha@xxxxxxxxxx
Phone: 919-254-1848 (T/L: 444)

Naci Dai <naci.dai@xxxxxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

02/24/2005 06:22 AM

Please respond to
wtp-dev


To
wtp-dev@xxxxxxxxxxx
cc
Subject
[wtp-dev] M3 and testing











The last build still has test errors in jst.jee, jst.servlet and
jst.validation.  I have tried to run these tests on my local machine and
they have failed in a different way.

-At this point if we think these tests pass but are not running due to a
configuration problem, we should pull the tests back and respin another
build.  All other criteria for decalring M3 are still valid.

-  If they are failing due to a valid problem in WTP then I guess we will
not declare M3 until after eclipsecon


Naci Dai,
Managing Director

eteration a.s.
Inonu cad. Sumer sok. Zitas D1-15
Kozyatagi, Istanbul 81090
+90 (532) 573 7783 (cell)
+90 (216) 361 5434 (phone)
+90 (216) 361 2034 (fax)
http://www.eteration.com
mailto:nacidai@xxxxxxx
mailto:naci@xxxxxxxxxxxxx



_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/wtp-dev



Back to the top