Bug 318282 - Problem in nodes removal/replacement for ASTRewrite.rewriteAST(...). Suspecting incorrect offsets calculation
Summary: Problem in nodes removal/replacement for ASTRewrite.rewriteAST(...). Suspecti...
Status: VERIFIED INVALID
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.7 M1   Edit
Assignee: Olivier Thomann CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-29 05:12 EDT by smakady CLA
Modified: 2010-08-03 07:30 EDT (History)
2 users (show)

See Also:


Attachments
The test file that gets screwed on certain nodes removal (16.59 KB, application/octet-stream)
2010-06-29 14:59 EDT, smakady CLA
no flags Details
A plugin project that shows the shows the bug (28.16 KB, application/octet-stream)
2010-07-01 14:26 EDT, smakady CLA
no flags Details
Patched version of writeCUDown(..) (9.87 KB, text/plain)
2010-07-15 10:30 EDT, Olivier Thomann CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description smakady CLA 2010-06-29 05:12:55 EDT
Build Identifier: M20100211-1343 

Hello,
I am using Eclipse JDT to remove certain statements (that have certain method invocations) within Java files. I am using ASTRewrite to do that. It works properly for some files, but is screws the AST syntax of some other files. Here is the code that I use to remove the method invocations:

ListRewrite list = rewrite.getListRewrite(s.getParent(), Block.STATEMENTS_PROPERTY); //where s is the statement to remove
list.remove(s, null);

And then, to update the Java file, I use the following code:

Hashtable options = JavaCore.getDefaultOptions();
options.put(DefaultCodeFormatterConstants.FORMATTER_PUT_EMPT Y_STATEMENT_ON_NEW_LINE,DefaultCodeFormatterConstants.TRUE);
org.eclipse.text.edits.TextEdit te = rewrite.rewriteAST(doc, options);
te.apply(doc);
file.setContents(new java.io.StringBufferInputStream(doc.get()), IResource.FORCE, null);

The problem happens when I try to remove all the assertions inside the method testFieldAndFormat() in this file (part of JabRef code):
public class UtilTest extends TestCase {

public void testNCase() {
assertEquals("", Util.nCase(""));
assertEquals("Hello world", Util.nCase("Hello World"));
assertEquals("A", Util.nCase("a"));
assertEquals("Aa", Util.nCase("AA"));
}

public void testGetPublicationDate(){

assertEquals("2003-02", Util.getPublicationDate(BibtexParser
.singleFromString("@ARTICLE{HipKro03, year = {2003}, month = #FEB# }")));

assertEquals("2003-03", Util.getPublicationDate(BibtexParser
.singleFromString("@ARTICLE{HipKro03, year = {2003}, month = 3 }")));

assertEquals("2003", Util.getPublicationDate(BibtexParser
.singleFromString("@ARTICLE{HipKro03, year = {2003}}")));

assertEquals(null, Util.getPublicationDate(BibtexParser
.singleFromString("@ARTICLE{HipKro03, month = 3 }")));

assertEquals(null, Util.getPublicationDate(BibtexParser
.singleFromString("@ARTICLE{HipKro03, author={bla}}")));

assertEquals("2003-12", Util.getPublicationDate(BibtexParser
.singleFromString("@ARTICLE{HipKro03, year = {03}, month = #DEC# }")));

}

public void testCheckName() {
assertEquals("aa.bib", Util.checkName("aa"));
assertEquals(".bib", Util.checkName(""));
assertEquals("a.bib", Util.checkName("a.bib"));
assertEquals("a.bib", Util.checkName("a"));
assertEquals("a.bb.bib", Util.checkName("a.bb"));
}

public void testCreateNeutralId() {

HashSet<String> set = new HashSet<String>();
for (int i = 0; i < 10000; i++){
String string = Util.createNeutralId();
assertFalse(set.contains(string));
set.add(string);
}

}

public void testPlaceDialog() {
Dialog d = new JDialog();
d.setSize(50, 50);
Container c = new JWindow();
c.setBounds(100, 200, 100, 50);

Util.placeDialog(d, c);
assertEquals(125, d.getX());
assertEquals(200, d.getY());

// Test upper left corner
c.setBounds(0,0,100,100);
d.setSize(200, 200);

Util.placeDialog(d, c);
assertEquals(0, d.getX());
assertEquals(0, d.getY());
}

public void testParseField() {

assertEquals("", Util.parseField(""));

// Three basic types (references, { } and " ")
assertEquals("#hallo#", Util.parseField("hallo"));
assertEquals("hallo", Util.parseField("{hallo}"));
assertEquals("bye", Util.parseField("\"bye\""));

// Concatenation
assertEquals("longlonglonglong", Util.parseField("\"long\" # \"long\" # \"long\" # \"long\""));

assertEquals("hallo#bye#", Util.parseField("{hallo} # bye"));
}

public void testShaveString() {

assertEquals(null, Util.shaveString(null));
assertEquals("", Util.shaveString(""));
assertEquals("aaa", Util.shaveString(" aaa\t\t\n\r"));
assertEquals("a", Util.shaveString(" {a} "));
assertEquals("a", Util.shaveString(" \"a\" "));
assertEquals("{a}", Util.shaveString(" {{a}} "));
assertEquals("{a}", Util.shaveString(" \"{a}\" "));
assertEquals("\"{a\"}", Util.shaveString(" \"{a\"} "));
}

public void testCheckLegalKey() {

assertEquals("AAAA", Util.checkLegalKey("AA AA"));
assertEquals("SPECIALCHARS", Util.checkLegalKey("SPECIAL CHARS#{\\\"}~,^"));
assertEquals("AeaeaAAA", Util.checkLegalKey("������"));
assertEquals("", Util.checkLegalKey("\n\t\r"));
}

public void testReplaceSpecialCharacters() {
// Shouldn't German � be resolved to Ae
assertEquals("AeaeaAAA", Util.replaceSpecialCharacters("������ "));
assertEquals("Hallo Arger", Util.replaceSpecialCharacters("Hallo Arger"));
}

public void testJoin() {
String[] s = "ab/cd/ed".split("/");
assertEquals("ab\\cd\\ed", Util.join(s, "\\", 0, s.length));

assertEquals("cd\\ed", Util.join(s, "\\", 1, s.length));

assertEquals("ed", Util.join(s, "\\", 2, s.length));

assertEquals("", Util.join(s, "\\", 3, s.length));

assertEquals("", Util.join(new String[]{}, "\\", 0, 0));
}

public void testStripBrackets() {
assertEquals("foo", Util.stripBrackets("[foo]"));
assertEquals("[foo]", Util.stripBrackets("[[foo]]"));
assertEquals("foo", Util.stripBrackets("foo]"));
assertEquals("foo", Util.stripBrackets("[foo"));
assertEquals("", Util.stripBrackets(""));
assertEquals("", Util.stripBrackets("[]"));
assertEquals("", Util.stripBrackets("["));
assertEquals("", Util.stripBrackets("]"));
assertEquals("f[]f", Util.stripBrackets("f[]f"));

try {
Util.stripBrackets(null);
fail();
} catch(NullPointerException npe){

}
}

BibtexDatabase database;
BibtexEntry entry;

public void setUp(){

StringReader reader = new StringReader(
"@ARTICLE{HipKro03," + "\n" + 
" author = {Eric von Hippel and Georg von Krogh}," + "\n" + 
" title = {Open Source Software and the \"Private-Collective\" Innovation Model: Issues for Organization Science}," + "\n" + 
" journal = {Organization Science}," + "\n" + 
" year = {2003}," + "\n" + 
" volume = {14}," + "\n" + 
" pages = {209--223}," + "\n" + 
" number = {2}," + "\n" + 
" address = {Institute for Operations Research and the Management Sciences (INFORMS), Linthicum, Maryland, USA}," + "\n" + 
" doi = {http://dx.doi.org/10.1287/orsc.14.2.209.14992}," + "\n" + 
" issn = {1526-5455}," + "\n" + 
" publisher = {INFORMS}" + "\n" + 
"}");

BibtexParser parser = new BibtexParser(reader); 
ParserResult result = null;
try {
result = parser.parse();
} catch (Exception e){
fail();
}
database = result.getDatabase();
entry = database.getEntriesByKey("HipKro03")[0];

assertNotNull(database);
assertNotNull(entry);
}

public void testParseMethodCalls(){

assertEquals(1, Util.parseMethodsCalls("bla").size());
assertEquals("bla", ((Util.parseMethodsCalls("bla").get(0)))[0]);

assertEquals(1, Util.parseMethodsCalls("bla,").size());
assertEquals("bla", ((Util.parseMethodsCalls("bla,").get(0)))[0]);

assertEquals(1, Util.parseMethodsCalls("_bla.bla.blub,").size());
assertEquals("_bla.bla.blub", ((Util.parseMethodsCalls("_bla.bla.blub,").get(0)))[0]);


assertEquals(2, Util.parseMethodsCalls("bla,foo").size());
assertEquals("bla", ((Util.parseMethodsCalls("bla,foo").get(0)))[0]);
assertEquals("foo", ((Util.parseMethodsCalls("bla,foo").get(1)))[0]);

assertEquals(2, Util.parseMethodsCalls("bla(\"test\"),foo(\"fark\")").size());
assertEquals("bla", ((Util.parseMethodsCalls("bla(\"test\"),foo(\"fark\")").get(0)))[0]);
assertEquals("foo", ((Util.parseMethodsCalls("bla(\"test\"),foo(\"fark\")").get(1)))[0]);
assertEquals("test", ((Util.parseMethodsCalls("bla(\"test\"),foo(\"fark\")").get(0)))[1]);
assertEquals("fark", ((Util.parseMethodsCalls("bla(\"test\"),foo(\"fark\")").get(1)))[1]);

assertEquals(2, Util.parseMethodsCalls("bla(test),foo(fark)").size());
assertEquals("bla", ((Util.parseMethodsCalls("bla(test),foo(fark)").get(0)))[0]);
assertEquals("foo", ((Util.parseMethodsCalls("bla(test),foo(fark)").get(1)))[0]);
assertEquals("test", ((Util.parseMethodsCalls("bla(test),foo(fark)").get(0)))[1]);
assertEquals("fark", ((Util.parseMethodsCalls("bla(test),foo(fark)").get(1)))[1]);
}


public void testFieldAndFormat(){
assertEquals("Eric von Hippel and Georg von Krogh", Util.getFieldAndFormat("[author]", entry, database));

assertEquals("Eric von Hippel and Georg von Krogh", Util.getFieldAndFormat("author", entry, database));

assertEquals(null, Util.getFieldAndFormat("[unknownkey]", entry, database));

assertEquals(null, Util.getFieldAndFormat("[:]", entry, database));

assertEquals(null, Util.getFieldAndFormat("[:lower]", entry, database));

assertEquals("eric von hippel and georg von krogh", Util.getFieldAndFormat("[author:lower]", entry, database));

assertEquals("HipKro03", Util.getFieldAndFormat("[bibtexkey]", entry, database));

assertEquals("HipKro03", Util.getFieldAndFormat("[bibtexkey:]", entry, database));
}

public void testUserFieldAndFormat(){

String[] names = Globals.prefs.getStringArray(NameFormatterTab.NAME_FORMATER_ KEY);
if (names == null)
names = new String[]{};

String[] formats = Globals.prefs.getStringArray(NameFormatterTab.NAME_FORMATTER _VALUE);
if (formats == null)
formats = new String[]{};

try {

List<String> f = new LinkedList<String>(Arrays.asList(formats));
List<String> n = new LinkedList<String>(Arrays.asList(names));

n.add("testMe123454321");
f.add("*@*@test");

String[] newNames = n.toArray(new String[n.size()]);
String[] newFormats = f.toArray(new String[f.size()]);

Globals.prefs.putStringArray(NameFormatterTab.NAME_FORMATER_ KEY, newNames);
Globals.prefs.putStringArray(NameFormatterTab.NAME_FORMATTER _VALUE, newFormats);

assertEquals("testtest", Util.getFieldAndFormat("[author:testMe123454321]", entry, database));

} finally {
Globals.prefs.putStringArray(NameFormatterTab.NAME_FORMATER_ KEY, names);
Globals.prefs.putStringArray(NameFormatterTab.NAME_FORMATTER _VALUE, formats);
}
}



public void testExpandBrackets(){

assertEquals("", Util.expandBrackets("", entry, database));

assertEquals("dropped", Util.expandBrackets("drop[unknownkey]ped", entry, database));

assertEquals("Eric von Hippel and Georg von Krogh", 
Util.expandBrackets("[author]", entry, database));

assertEquals("Eric von Hippel and Georg von Krogh are two famous authors.", 
Util.expandBrackets("[author] are two famous authors.", entry, database));

assertEquals("Eric von Hippel and Georg von Krogh are two famous authors.", 
Util.expandBrackets("[author] are two famous authors.", entry, database));

assertEquals("Eric von Hippel and Georg von Krogh have published Open Source Software and the \"Private-Collective\" Innovation Model: Issues for Organization Science in Organization Science.", 
Util.expandBrackets("[author] have published [title] in [journal].", entry, database));
}

public void testSanitizeUrl() {

assertEquals("http://www.vg.no", Util.sanitizeUrl("http://www.vg.no"));
assertEquals("http://www.vg.no/fil%20e.html",
Util.sanitizeUrl("http://www.vg.no/fil e.html"));
assertEquals("http://www.vg.no/fil%20e.html",
Util.sanitizeUrl("http://www.vg.no/fil%20e.html"));
assertEquals("www.vg.no/fil%20e.html",
Util.sanitizeUrl("www.vg.no/fil%20e.html"));

assertEquals("www.vg.no/fil%20e.html",
Util.sanitizeUrl("\\url{www.vg.no/fil%20e.html}"));

/**
* DOI Test cases
*/
assertEquals("http://dx.doi.org/10.1109/VLHCC.2004.20", Util.sanitizeUrl("10.1109/VLHCC.2004.20"));
assertEquals("http://dx.doi.org/10.1109/VLHCC.2004.20", Util.sanitizeUrl("doi://10.1109/VLHCC.2004.20"));
assertEquals("http://dx.doi.org/10.1109/VLHCC.2004.20", Util.sanitizeUrl("doi:/10.1109/VLHCC.2004.20"));
assertEquals("http://dx.doi.org/10.1109/VLHCC.2004.20", Util.sanitizeUrl("doi:10.1109/VLHCC.2004.20"));

/**
* Additional testcases provided by Hannes Restel and Micha Beckmann.
*/
assertEquals("ftp://www.vg.no", Util.sanitizeUrl("ftp://www.vg.no"));
assertEquals("file://doof.txt", Util.sanitizeUrl("file://doof.txt")); 
assertEquals("file:///", Util.sanitizeUrl("file:///"));
assertEquals("/src/doof.txt", Util.sanitizeUrl("/src/doof.txt"));
assertEquals("/", Util.sanitizeUrl("/"));
assertEquals("/home/user/example.txt", Util.sanitizeUrl("/home/user/example.txt"));
}

public void test2to4DigitsYear(){
assertEquals("1990", Util.toFourDigitYear("1990"));
assertEquals("190", Util.toFourDigitYear("190"));
assertEquals("1990", Util.toFourDigitYear("90", 1990));
assertEquals("1990", Util.toFourDigitYear("90", 1991));
assertEquals("2020", Util.toFourDigitYear("20", 1990));
assertEquals("1921", Util.toFourDigitYear("21", 1990));
assertEquals("1922", Util.toFourDigitYear("22", 1990));
assertEquals("2022", Util.toFourDigitYear("22", 1992));
assertEquals("1999", Util.toFourDigitYear("99", 2001));
assertEquals("1931", Util.toFourDigitYear("1931", 2001));
assertEquals("2031", Util.toFourDigitYear("31", 2001));
assertEquals("1932", Util.toFourDigitYear("32", 2001));
assertEquals("1944", Util.toFourDigitYear("44", 2001));
assertEquals("2011", Util.toFourDigitYear("11", 2001));

int thisYear = Calendar.getInstance().get(Calendar.YEAR);
int d2 = thisYear % 100;

NumberFormat f = new DecimalFormat("00");

for (int i = 0; i <= 30; i++){
assertTrue("" + i, thisYear <= Integer.parseInt(Util.toFourDigitYear(f.format((d2 + i) % 100))));
}
for (int i = 0; i < 70; i++){
assertTrue("" + i, thisYear >= Integer.parseInt(Util.toFourDigitYear(f.format((d2 - i + 100) % 100))));
}
}

public void testToMonthNumber(){

assertEquals(0, Util.getMonthNumber("jan"));
assertEquals(1, Util.getMonthNumber("feb"));
assertEquals(2, Util.getMonthNumber("mar"));
assertEquals(3, Util.getMonthNumber("apr"));
assertEquals(4, Util.getMonthNumber("may"));
assertEquals(5, Util.getMonthNumber("jun"));
assertEquals(6, Util.getMonthNumber("jul"));
assertEquals(7, Util.getMonthNumber("aug"));
assertEquals(8, Util.getMonthNumber("sep"));
assertEquals(9, Util.getMonthNumber("oct"));
assertEquals(10,Util.getMonthNumber("nov"));
assertEquals(11,Util.getMonthNumber("dec"));

assertEquals(0, Util.getMonthNumber("#jan#"));
assertEquals(1, Util.getMonthNumber("#feb#"));
assertEquals(2, Util.getMonthNumber("#mar#"));
assertEquals(3, Util.getMonthNumber("#apr#"));
assertEquals(4, Util.getMonthNumber("#may#"));
assertEquals(5, Util.getMonthNumber("#jun#"));
assertEquals(6, Util.getMonthNumber("#jul#"));
assertEquals(7, Util.getMonthNumber("#aug#"));
assertEquals(8, Util.getMonthNumber("#sep#"));
assertEquals(9, Util.getMonthNumber("#oct#"));
assertEquals(10,Util.getMonthNumber("#nov#"));
assertEquals(11,Util.getMonthNumber("#dec#"));

assertEquals(0, Util.getMonthNumber("January"));
assertEquals(1, Util.getMonthNumber("February"));
assertEquals(2, Util.getMonthNumber("March"));
assertEquals(3, Util.getMonthNumber("April"));
assertEquals(4, Util.getMonthNumber("May"));
assertEquals(5, Util.getMonthNumber("June"));
assertEquals(6, Util.getMonthNumber("July"));
assertEquals(7, Util.getMonthNumber("August"));
assertEquals(8, Util.getMonthNumber("September"));
assertEquals(9, Util.getMonthNumber("October"));
assertEquals(10,Util.getMonthNumber("November"));
assertEquals(11,Util.getMonthNumber("Decembre"));

assertEquals(0, Util.getMonthNumber("01"));
assertEquals(1, Util.getMonthNumber("02"));
assertEquals(2, Util.getMonthNumber("03"));
assertEquals(3, Util.getMonthNumber("04"));
assertEquals(4, Util.getMonthNumber("05"));
assertEquals(5, Util.getMonthNumber("06"));
assertEquals(6, Util.getMonthNumber("07"));
assertEquals(7, Util.getMonthNumber("08"));
assertEquals(8, Util.getMonthNumber("09"));
assertEquals(9, Util.getMonthNumber("10"));

assertEquals(0, Util.getMonthNumber("1"));
assertEquals(1, Util.getMonthNumber("2"));
assertEquals(2, Util.getMonthNumber("3"));
assertEquals(3, Util.getMonthNumber("4"));
assertEquals(4, Util.getMonthNumber("5"));
assertEquals(5, Util.getMonthNumber("6"));
assertEquals(6, Util.getMonthNumber("7"));
assertEquals(7, Util.getMonthNumber("8"));
assertEquals(8, Util.getMonthNumber("9"));

assertEquals(10,Util.getMonthNumber("11"));
assertEquals(11,Util.getMonthNumber("12"));

assertEquals(-1,Util.getMonthNumber(";lkjasdf"));
assertEquals(-1,Util.getMonthNumber("3.2"));
assertEquals(-1,Util.getMonthNumber("#test#"));
assertEquals(-1,Util.getMonthNumber(""));
}

public void testToUpperCharFirst(){

assertEquals("", Util.toUpperFirstLetter(""));
assertEquals("A", Util.toUpperFirstLetter("a"));
assertEquals("A", Util.toUpperFirstLetter("A"));
assertEquals("An", Util.toUpperFirstLetter("an"));
assertEquals("AN", Util.toUpperFirstLetter("AN"));
assertEquals("TestTest", Util.toUpperFirstLetter("testTest"));

}
}

=========================================================================
What happens is that the method testFieldAndFormat turns to look like :

public void testFieldAndForse));
}

I also tried replacing all the statements (having method invocations) in the testFieldAndFormat test case, by equivalent statements within block comments (e.g. /* methodInvocation */), but the new comment would be inserted 5 positions earlier than its expected offset within the test method's body .The output would look like this:
	public void testFieldAndFormat(/** Removed node:
		
	){
		assertEquals("Eric von Hippel and Georg von Krogh", Util.getFieldAndFormat("[author]", entry, databa
	
	*/se));
/** Removed node:	
				
		assertEquals("Eric von Hippel and Georg von Krogh", Util.getFieldAndFormat("author", entry, databa
		
		*/se));
/** Removed node:		
				
		assertEquals(null, Util.getFieldAndFormat("[unknownkey]", entry, databa
		
		*/se));
/** Removed node:		
				
		assertEquals(null, Util.getFieldAndFormat("[:]", entry, databa
		
		*/se));
/** Removed node:		
				
		assertEquals(null, Util.getFieldAndFormat("[:lower]", entry, databa
		
		*/se));
/** Removed node:		
				
		assertEquals("eric von hippel and georg von krogh", Util.getFieldAndFormat("[author:lower]", entry, databa
		
		*/se));
/** Removed node:		
				
		assertEquals("HipKro03", Util.getFieldAndFormat("[bibtexkey]", entry, databa
		
		*/se));
/** Removed node:		
				
		assertEquals("HipKro03", Util.getFieldAndFormat("[bibtexkey:]", entry, databa
		
		*/



Reproducible: Always

Steps to Reproduce:
1.Find all the statements that have method invocations in the test method testFieldAndFormat in JabRef (it works good for some other classes, so it needs to be that specific file)
2. Try to delete/replace all those found statements.
3. The syntax and structure of that test method will be screwed as explained in the details above.
Comment 1 Srikanth Sankaran CLA 2010-06-29 11:02:46 EDT
Please follow up, Thanks.
Comment 2 Olivier Thomann CLA 2010-06-29 11:07:47 EDT
Could you please attach the test case rather than pasting it in?
That would preserve the correct encoding for source files ?
What encoding are you using ?
Thanks.
Comment 3 smakady CLA 2010-06-29 14:59:38 EDT
Created attachment 173035 [details]
The test file that gets screwed on certain nodes removal

I am attaching the test case file that is causing the problem. I set the Eclipse workspace encoding to UTF-8 for all the java files, so this Java file is opened in UTF-8 within Eclipse when I run my code (which causes the problems). Does that answer your question about encoding?
Comment 4 Olivier Thomann CLA 2010-06-29 15:08:42 EDT
yes, this is fine. Now that I have the test case, I'll try to reproduce once I get the code that is calling:
ListRewrite list = rewrite.getListRewrite(s.getParent(),
Block.STATEMENTS_PROPERTY); //where s is the statement to remove
list.remove(s, null);

Thanks.
Comment 5 smakady CLA 2010-06-29 16:16:52 EDT
In fact the code size it large, so I tried to put the part that is directly preceding the node removal to make it more readable, and decrease the confusion.

The idea is that I look for certain method invocations, retrieve their parent statements, and then remove those statements, or comment them out.

The test method that I wanted to apply that idea on, was "testFieldAndFormat", and the invocations that I wanted to remove were for Util.getFieldAndFormat(..), which is invoked 8 times within testFieldAndFormat. I successfully captured the 8 statements containing those 8 method invocations, but the problem happens on deletion.

I am pasting that code below. Thanks.



import java.util.ArrayList;
import java.util.Vector;
import org.eclipse.jdt.core.IMember;
import org.eclipse.jdt.core.IMethod;
import org.eclipse.jdt.core.dom.ASTNode;
import org.eclipse.jdt.core.dom.ASTVisitor;
import org.eclipse.jdt.core.dom.Block;
import org.eclipse.jdt.core.dom.IMethodBinding;
import org.eclipse.jdt.core.dom.IfStatement;
import org.eclipse.jdt.core.dom.MethodInvocation;
import org.eclipse.jdt.core.dom.Statement;
import org.eclipse.jdt.core.dom.rewrite.ASTRewrite;
import org.eclipse.jdt.core.dom.rewrite.ListRewrite;

public class FindAndRemoveMethodCalls {
	
	private static final String REWRITE = "NODE_REWRITE";
	
	public void methodB(ASTNode sourceMethodNode /*Stands for the test method to be modified */, ASTNode node /*Stands for the TestClass's ASTNode*/, String callToRemove, final IMember targetMemberToRemove)
	{
		final Vector<ASTNode> matchedInvocations = new Vector<ASTNode>();
		final ArrayList<MethodInvocation> removedMethodInvocationNodes = new ArrayList<MethodInvocation>();
		Vector<ASTNode> doneContainers = new Vector<ASTNode>();
	
		// null SMN just means the method isn't in the node
		if (sourceMethodNode != null)
			sourceMethodNode.accept(new ASTVisitor() {

				
				public boolean visit(MethodInvocation node) {

					IMethodBinding mBinding = node.resolveMethodBinding();

					if (mBinding == null)
					{
						;
					}
					if (mBinding != null)
						// A user-defined comparison to know if the current invocation matches the target call to remove
						//For the test method testFieldAndFormat, I wanted to remove all the statements that include invocations of Util.getFieldAndFormat(..) method
							if (ModelUtilities.matchMethod(mBinding, (IMethod) targetMemberToRemove)) 
							{
								matchedInvocations.add(node);
								removedMethodInvocationNodes.add(node);
							}
						
					return super.visit(node);
				}

			});

		ASTRewrite rewrite = (ASTRewrite) node.getProperty(REWRITE);
		/*This property "REWRITE" is set in an earlier part of the code by the following:
		ASTRewrite newRewrite = ASTRewrite.create(node.getAST());		
		*/
		
		if(matchedInvocations.isEmpty())
			return;
		
		//The code works properly till here, where it captures all the method invocations inside the testFieldAndFormat test case
		for (ASTNode n : matchedInvocations) {
			
			
			Statement s = getContainingStatement(n);
			if (!doneContainers.contains(s)) {	
				doneContainers.add(s);

				if (s instanceof IfStatement)
					;
				
						//Option 1
				//START: Code for node comment out
				ASTNode placeHolder = rewrite.createStringPlaceholder(
						"/** Removed node: \n",
						ASTNode.EMPTY_STATEMENT);
				ListRewrite list = rewrite.getListRewrite(s.getParent(), Block.STATEMENTS_PROPERTY);
				list.insertBefore(placeHolder, s, null);

				placeHolder = rewrite.createStringPlaceholder("\n*/", ASTNode.EMPTY_STATEMENT);
				list = rewrite.getListRewrite(s.getParent(), Block.STATEMENTS_PROPERTY);
				list.insertAfter(placeHolder, s, null);
				//END code for node comment out
				
						//Option 2
				//START: Code for node removal
				//ListRewrite list = rewrite.getListRewrite(s.getParent(),Block.STATEMENTS_PROPERTY); //where s is the statement to remove
				//list.remove(s, null);
				//END: Code for node removal
				
				
			}
	
		}

}

private Statement getContainingStatement(ASTNode n) {
	if (n instanceof Statement)
	return (Statement) n;
	else
	return getContainingStatement(n.getParent());
	}

}
Comment 6 smakady CLA 2010-06-29 17:17:12 EDT
I just want to add that I am using Eclipse version: 3.5.2, and JDT version: 3.5.2.v_981_R35x, not 3.7.
Comment 7 Olivier Thomann CLA 2010-06-30 13:38:05 EDT
What is the code for:
ModelUtilities.matchMethod(mBinding,(IMethod) targetMemberToRemove) ?

Thanks.
Comment 8 Olivier Thomann CLA 2010-06-30 13:49:30 EDT
Would it be possible to speed up the resolution of this bug to get a self-containing example that produces the wrong output?
So some code that calls the methodB(...) and also all the necessary code to compile the given code snippet (in comment 5).
Thanks.
Comment 9 smakady CLA 2010-07-01 14:26:38 EDT
Created attachment 173249 [details]
A plugin project that shows the shows the bug

Hello,
Attached, please find a plug-in project that I created and that shows the bug. The project has one view "ASTBug view" in the "ASTBug" category. In that view, there is one action called "BugAction". The project assumes that you have JabRef2.5 project source code (and test code) imported in the runtime eclipse instance. Also, the project assumes that the source code and test code are grouped in the same packages. For instance, UtilTest.java will be in the package net.sf.jabref, not in the package test.net.sf.jabref

Once you click on the "Bug action", it does the following:
1. Searches for the "UtilTest.java" compilation unit
2. Located the MethodDeclaration node for "testFieldAndFormat" test method
3. Retrieves all the method invocations for "getFieldAndFormat".
4. Removes those invocations.
Comment 10 smakady CLA 2010-07-07 17:58:15 EDT
Hello,
I just want to know if you managed to reproduce the bug or not, and if this problem would it be a high priority to solve. If it would not be solved soon, is there a certain workaround to be able to remove nodes without screwing the file's AST?

Thanks.
Comment 11 Olivier Thomann CLA 2010-07-12 12:11:05 EDT
I'll work on it this week. I had a couple of days off.
Comment 12 Olivier Thomann CLA 2010-07-15 09:56:12 EDT
Reproduced. I am investigating.
Comment 13 Olivier Thomann CLA 2010-07-15 10:30:21 EDT
Created attachment 174402 [details]
Patched version of writeCUDown(..)

Your problem is in the writeCUDown(..). When you retrieve the source of the file, you don't take the encoding into account so you apply the rewrite on a different source than the source used to create the rewrite. This leads to the problem you are having with a corrupted source.
The attached version works, but it hardcoded the encoding of the file.

I recommend that you save the ICompilationUnit source when you create the CompilationUnit node. Like this you will make sure that both are consistent.

I will close as INVALID as the problem doesn't come from ASTRewrite.
Comment 14 Olivier Thomann CLA 2010-07-15 10:30:47 EDT
Closing as INVALID. Let me know if you have any question.
Comment 15 smakady CLA 2010-07-20 18:44:15 EDT
Thanks a lot for your detailed reply. It helped a lot. I just have a quick question: If I run into similar problems like that, what would be the appropriate forum to post to (so that I don't open it as a bug). For this one, I posted it before to this forum (http://www.eclipse.org/forums/index.php?t=thread&frm_id=13&) but I don't if it was the correct forum, or there is a more specific forum.

Many thanks for your help.
Comment 16 Olivier Thomann CLA 2010-07-20 22:39:27 EDT
(In reply to comment #15)
> Thanks a lot for your detailed reply. It helped a lot. I just have a quick
> question: If I run into similar problems like that, what would be the
> appropriate forum to post to (so that I don't open it as a bug). For this one,
> I posted it before to this forum
> (http://www.eclipse.org/forums/index.php?t=thread&frm_id=13&) but I don't if it
> was the correct forum, or there is a more specific forum.
This is the right place to ask questions. I just haven't checked it for a while.
I'll check it more often starting today.

> Many thanks for your help.
You're welcome.

In general, make sure you are making modifications on the same source than the one used to create the AST.
Comment 17 Satyam Kandula CLA 2010-08-03 07:30:13 EDT
Verified for 3.7M1