Community
Participate
Working Groups
The attached testcase shows a construct which is blindly accepted by eclipse but rejected by other popular tools on the market. Given eclipse track record at letting restriction errors pas I tend to trust the other tools more, but I didn't dissect the W3C spec to check which one of them is actually right
Created attachment 44395 [details] testcase
Eclipse SDK Version: 3.2.0 Build id: I20060512-1600 WebTools 1.5 RC3 Windows 2000
Lawrence, bring it up in the sample xsd editor and you'll see the error.
Valentin, please reassign if you're not the owner of the XSD validator.
The WTP XSD validator is based on Xerces and sets this Xerces feature http://apache.org/xml/features/validation/schema-full-checking explicitly to false (even though false is the default). See [1] for more information about the feature. If I set the feature to true, then the validator produces the following messages: Severity and Description Path Resource Location Creation Time Id derivation-ok-restriction.5.4.2: Error for type 'RestrictedFoo'. The particle of the type is not a valid restriction of the particle of the base. 147033 test2.xsd line 35 1179349296187 48 rcase-Recurse.2: There is not a complete functional mapping between the particles. 147033 test2.xsd line 35 1179349296187 47 Craig, I assume there was a specific reason why this was set to false. I'm thinking for 3.0 we can make this a preference, similar to honour-all-schemaLocations? [1] http://xerces.apache.org/xerces-j/features.html#schema-full-checking
We turned off 'full checking' since it caused serious perfomance degredation for some use cases (cc'ing Lawrence since he knows the full story). BTW, I've heard the Xerces team may have fixed this problem in version 2.9.0. So Nicolas, perhaps you could do us the service of replacing the xerces jars included in WTP's xerces plugin with this 2.9.0 version and change the following code in the XSDValidator class (in the xsd.core plugin) grammarPreparser.setFeature(Constants.XERCES_FEATURE_PREFIX + Constants.SCHEMA_FULL_CHECKING, true);// false Then you could verify if this issue is fixed with latest Xerces. Your help would be very appreciated!
*** Bug 146832 has been marked as a duplicate of this bug. ***
My personal preference on this issue, is to have a preference page that would allow the user to decide if full schema checking should be done. If dealing with standard schemas for B2B checking, full schema checking should always be done. Performance can be improved, by making sure the schemas are cached the first time they are loaded.
This bug 185839 deals with a proposed upgrade to Xerces 2.9. I'm not entirely sure when that will happen though. We already have a preference page for XML Schema files in org.eclipse.wst.xsd.ui. Currently it holds mostly editor specific settings plus the honour all schema locations validator preference as the lone validation preference. Ideally we would have a preference page with settings for the XSD validator to give the user a chance to tweak more settings but that may not happen for a while. So, if having this switch is really important (there have been a few other questions on this, see the dups) I guess we could add a new check box alongside the one for the honour all schema location I would suggest leaving the setting to false by default until the upgrade to Xerces 2.9 happens and we verify that the performance gains are there. Since the change is not very complicated I'll mark as help wanted. Are you willing to take a first crack at a patch David?
(In reply to comment #9) > So, if having this switch is really important (there have been a few other > questions on this, see the dups) I guess we could add a new check box alongside > the one for the honour all schema location I would suggest leaving the setting > to false by default until the upgrade to Xerces 2.9 happens and we verify that > the performance gains are there. > > Since the change is not very complicated I'll mark as help wanted. Are you > willing to take a first crack at a patch David? Mark it for Bug Day and I'll take a crack at it, as I do need to have full compliance checking enabled so that my users aren't getting false positives in validation checking.
I'll add a check box and appropriate code to make this an optional setting. I think the default value of this check box should be true. The reasoning is that it will guarantee the greatest level of conformance across schema parsers. If lax validation checking on an XSD is allowed by default, this can generate false positives that the implementor may not be expecting. We should really check to make sure that Xerces 2.9.1 is going to be included as in addition to some performance and XSD issues that were introduced in 2.8.x and fixed in 2.9.x, 2.9.x is also undergoing testing of the W3C XML Schema test suite and has addressed some fringe issues already. I should have a patch to post in a few days.
Created bug 204361 for the request to upgrade Orbit's Xerces bundle to 2.9.1, so that additional XSD related issues can be addressed.
Created attachment 79034 [details] Allow Toggling of Full XSD Conformance for XSD Validation The current implementation of the XSD Editor and Validators do not enforce full XSD Conformance checking. This will bypass the Identity Constraint checking which can lead to false positives on the schema. This patch addresses this issue by adding a check box to the XSD Preferences page so that this is user selectable. It currently is set to true by default, but can be turned off if the user desires.
Comment on attachment 79034 [details] Allow Toggling of Full XSD Conformance for XSD Validation Removed bad patch
Created attachment 79035 [details] Corrected Patch Corrects previous patch.
Created attachment 79037 [details] Adds option for control Full Schema Validation Corrected patch that adds the conformance preference and makes the XSD Validator look and use the conformance preference setting.
Looking further into this...it appears that the WSDL editor when validating the inline XSD doesn't call the same validator as the XSD editor. The issue here is that it means seperate settings are being used, and duplicate code appears to be maintained. Some refactoring on this might be needed going forward to maintain consistency.
Thanks David, I'll check out the patch. As I said, until we can be certain that using a default of true does not significantly degrade performance I'd prefer we stick with a default of false. One way to check this is by running the XSD performance tests - which I believe you are familiar with. Would you mind also maybe adding a new JUnit for this scenario to the org.eclipse.wst.xsd.validation.tests plug-in. I would probably suggest the BugFixes tests suite. There's an "honour all schema location" test in there so you can probably model a new test after that. Pick one or more scenarios that are only showing up when the new option is on and follow the bread crumbs :-). Regarding the WSDL validator, you're right, it would be good if it was consistent. Refactoring is the ideal way to maintain consistency although we have to make sure there are no other valid reasons for the duplication. Lawrence, are there any reasons why refactoring may not be recomended here? I'm thinking maybe you were trying to minimize dependencies between the XSD and WSDL validators?
The WSDL validator can run outside of Eclipse. Keeping it simple (few dependencies) and without Eclipse dependencies is key for this function. Also, there is some custom XML schema validation code required to handle inline schema validation although I expect that this could be factored out.
(In reply to comment #19) > The WSDL validator can run outside of Eclipse. Keeping it simple (few > dependencies) and without Eclipse dependencies is key for this function. Also, > there is some custom XML schema validation code required to handle inline > schema validation although I expect that this could be factored out. > Personally, I think the WSDL validator if it validates the XSDs should have a dependency on the XSD validator. As it is now.. you could potentially be fixing bugs in multiple places, or in the case of the above patch potentially add a full schema conformance check to the WSDL validator (currently it also does lax schema checking instead of full conformance checking which leads to false positives that the WSDL and XSDs included are correct). Again, I think Xerces 2.8.1 going forward addressed a lot of performance issues in the full schema checking arena. I'll create a JUnit test to go along with the patch to test the validation scenario and compliance. I might be able to find something from the W3C schema test suite that could be used as input to the tests. The full conformance in xerces enables identity constraint and key constraint checking of the schema itself.
From a user point of view, it is very distressing to have the same schemas behave differently when validated alone and when validated through and wsdl import. Please use a single validating path and make is as anal as necessary. When tools disagree, especially when two parts of the same tool disagree, humans waste hours trying to sort which one is right.
Dave and Nicolas - Thanks for your comments. I agree that there should be one schema validator. I too have experienced inconsistencies in my tools and find it extremely irritating. The WSDL validator should make use of the XML schema validator for inline schema validation. As I mentioned in comment #19, there are other requirements on the WSDL validator and we need to ensure this change is done without breaking existing usage scenarios. I think this has entered into new bug territory. Would one of you care to open a bug to voice your frustration?
Created bug 204488 to address the refactoring that should occur.
David, I assume the patch you attached is for the 3.0 stream right? Have you made any progress with the unit test? Do you need any information/help from me?
(In reply to comment #24) > David, I assume the patch you attached is for the 3.0 stream right? Have you > made any progress with the unit test? Do you need any information/help from me? > Yes, patch is against the Head branch in CVS. Haven't had a chance to get to the junit test, maybe tomorrow. Been swamped with the job that pays the bills.
Created attachment 79410 [details] JUnit Test for Full Conformance Patch Adds a Junit Test in the BugFixesTest.java for the Full XML Schema Conformance checking patch. This test uses as input the sample file that is attached to this bug.
Good stuff, Dave. We are in lock down mode to declare M2 so I am tentatively targetting this for early M3.
Nitin, please review, commit and release for WTP 3.0. Thank you.
Released for M3 with some changes: 1) The checkboxes on the preference page are in a single column. 2) The GridData objects are built using the recommended SWT constants instead of GridData's, and using separate instances for the two checkboxes (as directed in the GridData javadoc). 3) Changed the label "Full XML Schema conformance check" to "Check full XML Schema conformance". Thanks for the patches, David.
Hi Dave, The fix for this bug sounds like a New & Noteworthy item (http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg05699.html) Would you be willing to write up a little blurb for this? It should just be a sentence or two describing the new feature (in this case, the new preferences) and screenshot. Here's an example: http://www.eclipse.org/webtools/development/news/3.0M3/javaee.php You can just add it as a comment here and attach a screenshot to this bug. If you can't do this by the end of day tomorrow (Tuesday) or if I don't hear from you by then, that's fine, I'll just go ahead and do it myself.
Amy, I might be able to get to it tonight, pretty swamped today though. If you don't get something from me, please go ahead and write it up.
Created attachment 82720 [details] Image of "Check Full Schema Conformance' fix Image showing new preference option for XSD validation.
Amy, hope this is what you wanted: The XSD Preference page now has a new option to toggle whether XSDs are checked for full conformance to the XSD specification. This will enforce tighter adherence to the XML schema specification. This option is enabled by default.
Verified in build S-3.0M3-20071112224942. Validation error appears in attached testcase (test2.xsd) when full conformance checking is enabled. Error disappears if checking is disabled.
(In reply to comment #33) > Amy, hope this is what you wanted: Looks good, thanks Dave!
Closing.
mass change to add 'contributed' keyword based on bugzilla query, please correct if that's not accurate (by marking patches as obsolete and removing the 'contributed' keyword.
