Bug 70738 - Scalability problem in compiling classes with default access level
Summary: Scalability problem in compiling classes with default access level
Status: RESOLVED WORKSFORME
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.0   Edit
Hardware: PC Windows 2000
: P3 normal (vote)
Target Milestone: 3.1 M1   Edit
Assignee: Kent Johnson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-07-23 11:47 EDT by Yevgeny CLA
Modified: 2005-01-11 11:02 EST (History)
0 users

See Also:


Attachments
The eclipse screen snapshots with errors. (268.00 KB, application/octet-stream)
2004-07-23 11:53 EDT, Yevgeny CLA
no flags Details
Example of a file generated by AdventNet mib compiler (31.94 KB, text/plain)
2004-07-28 11:16 EDT, Yevgeny CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yevgeny CLA 2004-07-23 11:47:21 EDT
When a project contains classes with a default access level and these classes 
are contained in files with different names than the class names, the 
correctness of the compilation depends on the number of such classes.

I created a test case for it. In order to run it, you need to create two 
projects under the same workspace. The first project will contain file 
Generator that I used to generate test files for the problem. The name of the 
project is not important. Put Generator class (below) into the project and 
compile it.
The second project I called package-test. It is the place where generated test 
files will be stored.
Generator class takes one parameter: number of files to create. When you run it 
with parameter 10, you will see 10 files created in the project. Compile them 
and you will not see any compilation error.
When I ran Generator with parameter 2000 (the default value) and compiled the 
result, I got lots of compilation errors that class names cannot be resolved. 
When I repeated the compilation without cleaning, all errors were gone. 
Even more funny thing happened when I ran Generator with parameter 1000. In 
that case, the compiler did not report any error, but the editor for the test 
files did.

I don't exclude that the limit for the compiler for such case depends on the 
machine configuration. So, you may need to increase the parameter of Generator.
I hope my test case will help.
Is there any way around the problem?


-- file Generator.java --
import java.io.*;


/**
 * 
 * Generates test for package wide classes compilation
 */
public class Generator
{
    /**
     * 
     * @param args first argument (optional) is number of files to generate
     *             second argument (optional) defines the shift expressed in 
percents.
     */
    public static void main(String[] args)
    {
        int count = 2000;   // Number of classes to generate
        int perc= 26;
  
        String currentDirectory = System.getProperty("user.dir");
        File generatorDir = new File(currentDirectory);
        String directory = generatorDir.getParent() + "/package-test/test";
    
        
        if(args != null && args.length > 0)
        {
            count = Integer.parseInt(args[0]);
        }
        if(args != null && args.length > 1)
        {
            perc = Integer.parseInt(args[1]);
        }
        
        int shift = perc * count / 100;
        try
        {
            File dir = new File(directory);
            if(!dir.exists())
            {
                dir.mkdirs();
            }
            
            // delete old files
            File[] list = dir.listFiles();
            for(int i = 0; i<list.length; i++)
            {
                list[i].delete();
            }
    
            FileReader rd = new FileReader("template.txt");
            char[] tmp = new char[10000];
            int size = rd.read(tmp);
            String template = new String(tmp,0,size);
            
        
            
            for (int i = 0; i < count; i++)
            {
                String firstClassName = "First_" + i;
                String secondClassName = "Second_" + i;
                String calledClassName = "First_" + (i+shift)%count;
                String anotherClassName = "First_" + (i+2*shift/3)%count;
                String someClassName ="Second_" +((i*i)>>10)%count;
        
                String res = template.replaceAll("FirstClass", firstClassName)
                    .replaceAll("SecondClass",secondClassName)
                    .replaceAll("CalledClass",calledClassName)
                    .replaceAll("AnotherClass",anotherClassName)
                    .replaceAll("SomeClass",someClassName);
                
                
                File file = new File(directory, "Test_" + i + ".java");
                if(file.exists())
                {
                    file.delete();
                }
                
                file.createNewFile();
                FileWriter writer = new FileWriter(file);
                writer.write(res);
                writer.close();
                
            }
        }
        catch (IOException ex)
        {
            System.out.println(ex);
        }

    }
}
Comment 1 Yevgeny CLA 2004-07-23 11:53:35 EDT
Created attachment 13565 [details]
The eclipse screen snapshots with errors.

The first snapshot shows no errors in compilation but the editor displays the
error. It is the case of 1000 files.
The second snapshot shows lots of compilation errors. This is case of 2000
files.
Comment 2 Kent Johnson CLA 2004-07-27 16:28:08 EDT
Yes we do compile source files in chunks of 1000 files for memory reasons... 
but the spec is very clear that secondary types (types defined in a file with 
another name) cannot be found from source lookups.

Only once all .class files have been created can any compiler expect to find 
such types.

A few questions for you:

1. Why are you defining secondary or helper types & then referencing them 
outside the file?

2. Why can you not define these types in their own file with the correct name?
Comment 3 Yevgeny CLA 2004-07-28 11:16:33 EDT
Created attachment 13647 [details]
Example of a file generated by AdventNet mib compiler

I agree that it is not a good style to put a class referenced from outside to a
file with a different name. At the same time, it is a valid construct. JAVAC
does not have any problem with it. IMO, you should mention such limitation
somewhere.
In my project, all such files are generated by a third party tool. I cannot
change it.
Is there any way to increase the compilable chunk of files through some
compiler property?
Comment 4 Kent Johnson CLA 2004-07-28 12:57:07 EDT
You're mistaken that JAVAC does not have a problem with this.

Its a very order dependent problem... so try putting the generated source 
files in their own project(s) & change your project to depend on them. Then 
they should be compiled before your code.

Currently there is no user configurable option for the 1000 file limit so try 
dividing up your project.
Comment 5 Kent Johnson CLA 2004-08-03 14:45:01 EDT
Please reopen if dividing your project into multiple projects does not solve 
the problem.

Projects are eclipse's way of controlling memory use & providing a manageable 
group of classes for a team to work on.