Bug 228464 - Annotation.getMemberValuePairs() empty for single attribute with empty value
Summary: Annotation.getMemberValuePairs() empty for single attribute with empty value
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: 3.4 RC1   Edit
Assignee: David Audel CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-04-23 13:18 EDT by Ernest Mah CLA
Modified: 2008-05-13 11:54 EDT (History)
2 users (show)

See Also:
jerome_lanneluc: review+


Attachments
Proposed fix (29.06 KB, patch)
2008-05-05 08:02 EDT, David Audel CLA
no flags Details | Diff
Updated patch (32.29 KB, patch)
2008-05-06 16:33 EDT, David Audel CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ernest Mah CLA 2008-04-23 13:18:15 EDT
Build ID: I20080409-1425

Steps To Reproduce:
For annotation such as:

@ABC(name=)

Calling Annotation.getMemberValuePairs() returns an empty list.

I was expecting to get a membervaluepair for name, with it's value as unknown.


More information:
Comment 1 BensonN CLA 2008-04-23 14:54:35 EDT
For the "IAnnotation" object for the following annotation, the length of the source range is 4, looks like the "(name=)" is ignored.
Comment 2 Ernest Mah CLA 2008-04-23 15:32:51 EDT
Thanks for the extra information Benson.

If the source range is off, then we have no way of understanding which part of the source we should update later when we programmatically want to update the annotation.  Unless you have a workaround for this, it would be a major blocker for us.  Updating severity for now.

Thanks.
Comment 3 Jerome Lanneluc CLA 2008-04-24 07:32:35 EDT
I'm surprised to see this request as this issue was already discussed in bug 79112 comment 47. This comment actually referred to bug 130778 where it was explicitly said that incorrect member-value pairs would not be recovered. 

Given the time remaining before 3.4M7 (2 days before the 3.4M7 warm up build), I cannot promise anything.
Comment 4 Ernest Mah CLA 2008-04-24 10:45:00 EDT
(In reply to comment #3)
> I'm surprised to see this request as this issue was already discussed in bug
> 79112 comment 47. This comment actually referred to bug 130778 where it was
> explicitly said that incorrect member-value pairs would not be recovered. 
> 
> Given the time remaining before 3.4M7 (2 days before the 3.4M7 warm up build),
> I cannot promise anything.
> 

Hi Jerome, apologies up front.  Our memories aren't as sharp as yours :).  I tried searching for relevant bugs for background information and failed to find good references. Looks like bug 130778 did indeed mention the similar scenario, though in that case there was no Annotation object returned, so it has improved.  I was hoping the model would try and represent the close as source as possible, so we wouldn't have to resort down to the AST level to parse it.

Any help you can provide Jerome is greatly appreciated.

Barring an IMemberValuePair object being returned, is it possible to have getSource() return the annotation including the "(name=)" part of the annotation.  Just that would be enough to get us past this issue.

If not, is there anything we can query from Annotation to tell us that something didn't get loaded into the model and thus we should go down to the AST level?

Thanks much!

Comment 5 Jerome Lanneluc CLA 2008-04-24 11:10:55 EDT
We will try to come up with something for M7 (even after the warm up if needed). David is currently investigating.
Comment 6 Jerome Lanneluc CLA 2008-04-24 11:59:42 EDT
Talking with David, it looks like we should be able to return an IMemberValuePair with a K_UNKNOWN value. Also the source of the annotation should be "@ABC(name=". Note the missing closing parenthesis.

However we are wondering what you would do in this case. You talked about using the AST (I assume this is using org.eclipse.jdt.core.dom.ASTParser). But the AST won't give you more information. So what would it bring you?
Comment 7 Ernest Mah CLA 2008-04-24 12:28:44 EDT
(In reply to comment #6)
> Talking with David, it looks like we should be able to return an
> IMemberValuePair with a K_UNKNOWN value. Also the source of the annotation
> should be "@ABC(name=". Note the missing closing parenthesis.
> 
> However we are wondering what you would do in this case. You talked about using
> the AST (I assume this is using org.eclipse.jdt.core.dom.ASTParser). But the
> AST won't give you more information. So what would it bring you?
> 

I had guessed that trying to use the ASTParser ourselves would allow us to locate full source information.  We wanted the start and end locations so that we could replace the annotation an attributes with a new string serialized from our own representation.

What would happen in the case the malformed attribute lived in the middle of well formed attributes?

e.g. @ABC(attr1="happy",name=, attr3="unhappy")

and the simpler case 
@ABC(name=,attr3="unhappy")

Comment 8 Jerome Lanneluc CLA 2008-04-25 07:56:54 EDT
(In reply to comment #7)
> I had guessed that trying to use the ASTParser ourselves would allow us to
> locate full source information.  We wanted the start and end locations so that
> we could replace the annotation an attributes with a new string serialized from
> our own representation.
I'm not sure I understand: the IMemberValuePair API doesn't provide source positions.

> What would happen in the case the malformed attribute lived in the middle of
> well formed attributes?
> 
> e.g. @ABC(attr1="happy",name=, attr3="unhappy")
Right now you get 1 member-value pair for 'attr1' using either the Java model (IAnnotation) or using the DOM/AST (NormalAnnotation).
We could improve this scenario in both cases, and return 2 member-value pairs: 1 for 'attr1', and one for 'name'. But that is the best we could do for 3.4.

> 
> and the simpler case 
> @ABC(name=,attr3="unhappy")
> 
Right now you get no member-value pairs in both cases. Again we could improve this scenario and return 1 member-value pair for 'name'.
Comment 9 Ernest Mah CLA 2008-04-25 10:32:45 EDT
(In reply to comment #8)
> (In reply to comment #7)
> > I had guessed that trying to use the ASTParser ourselves would allow us to
> > locate full source information.  We wanted the start and end locations so that
> > we could replace the annotation an attributes with a new string serialized from
> > our own representation.
> I'm not sure I understand: the IMemberValuePair API doesn't provide source
> positions.

To clarify, we get the source range from the Annotation object by calling getSourceRange()and then use the offset and length to replace the entire annotation.  If this part could return the right parenthesis, this would solve the bigger problem for us. Any other suggestions you can provide for us to get around this?

The other changes to return IMemberValuePairs up to and including the last one including the name=, will help as well.

Thanks 
Comment 10 Jerome Lanneluc CLA 2008-04-28 07:41:12 EDT
(In reply to comment #9)
> To clarify, we get the source range from the Annotation object by calling
> getSourceRange()and then use the offset and length to replace the entire
> annotation.  If this part could return the right parenthesis, this would solve
> the bigger problem for us. Any other suggestions you can provide for us to get
> around this?
I just talked to David, and it looks like we won't be able to give you the right parenthesis in the source range for 3.4. This would be too big of a change given that our Release Candidate 0 is at the end of the week.
Unfortunately, all we can suggest is for you to scan the source for the right parenthesis. This is assuming that your scenarios are as simple as you described in this bug, i.e. 
- they don't involve annotations in annotation
  e.g. @ABC(name=, attr=@DEF()) would be less easy to scan.
- they don't involve complex expressions for the value
  e.g. @ABC(name=, attr=3*(2+4)) would be less easy to scan.

> The other changes to return IMemberValuePairs up to and including the last one
> including the name=, will help as well.
This change is less involved, and we should be able to release it during the RC1 week. Just let us know if you want us to release it.
Comment 11 Ernest Mah CLA 2008-04-28 16:33:36 EDT
(In reply to comment #10)

> I just talked to David, and it looks like we won't be able to give you the
> right parenthesis in the source range for 3.4. This would be too big of a
> change given that our Release Candidate 0 is at the end of the week.
> Unfortunately, all we can suggest is for you to scan the source for the right
> parenthesis. This is assuming that your scenarios are as simple as you
> described in this bug, i.e. 
> - they don't involve annotations in annotation
>   e.g. @ABC(name=, attr=@DEF()) would be less easy to scan.
> - they don't involve complex expressions for the value
>   e.g. @ABC(name=, attr=3*(2+4)) would be less easy to scan.
> 
Unfortunately we are trying to handle all cases including the ones you have described.  We will have to introduce some logic here to handle.

> > The other changes to return IMemberValuePairs up to and including the last one
> > including the name=, will help as well.
> This change is less involved, and we should be able to release it during the
> RC1 week. Just let us know if you want us to release it.
> 
Yes, this would be good for us.  Thanks Jerome.
Comment 12 David Audel CLA 2008-05-05 08:02:07 EDT
Created attachment 98618 [details]
Proposed fix

This fix only add recovery of the first incorrect member value pair.

For '@ABC(attr1="",attr2=,attr3="")' only attr1 and attr2 will be recovered.

The DOM AST will be @ABC(attr1="",attr2=$missing$) and the source range of the annotation node will include only the following characters: '@ABC(attr1="",attr2='.

In the java model the IMemberValuePair#getValueKind() will return K_UNKNOWN for the second member value pair and getValue() will return null. IAnnotation#getSource() will return '@ABC(attr1="",attr2='.
Comment 13 David Audel CLA 2008-05-05 09:11:58 EDT
Frederic - Could you review the patch ?
Comment 14 David Audel CLA 2008-05-05 10:32:02 EDT
Jerome - Could you review my patch ?
Comment 15 Jerome Lanneluc CLA 2008-05-06 06:07:29 EDT
It would be good to also check if the SingleNameReference is a fake identifier in the local variable case. See LocalVariable.getAnnotationMemberValue(MemberValuePair, Expression, JavaElement)
Comment 16 David Audel CLA 2008-05-06 16:33:40 EDT
Created attachment 98946 [details]
Updated patch

I updated the method LocalVariable.getAnnotationMemberValue() to take into account the fake identifier case and added a regression test for the annotations of a ILocalVariable returned by a search request.
Comment 17 Jerome Lanneluc CLA 2008-05-07 07:11:29 EDT
Patch looks good
Comment 18 David Audel CLA 2008-05-07 07:39:07 EDT
Released for 3.4RC1.

Tests added
  CompilationUnitTests#testAnnotations27()
  JavaSearchBugsTests#testBug228464()
  AnnotationDietRecoveryTest#test0038() -> test0039()

Tests updated
  ASTConverterBugsTestJLS3#testBug130778i() -> testBug130778s()
  StatementRecoveryTest_1_5#test0006()
  AnnotationDietRecoveryTest#test0018(), test0019(), test0021(), test0023(), test0035()
Comment 19 David Audel CLA 2008-05-07 07:52:08 EDT
Annotation recovery could be improved by using an algorithm similar to statements recovery but it is too risky for 3.4. I added the bug 230872 to keep trace of this possible improvement.
Comment 20 Jerome Lanneluc CLA 2008-05-13 11:54:45 EDT
Verified for 3.4RC1 using I20080510-2000