Bug 107079 - [1.5][compiler] mis-compiled Wildcard capture leads to a ClassCastException
Summary: [1.5][compiler] mis-compiled Wildcard capture leads to a ClassCastException
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows 2000
: P3 normal (vote)
Target Milestone: 3.1.1   Edit
Assignee: Philipe Mulet CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-08-15 19:15 EDT by Daniel Burke CLA
Modified: 2005-09-26 11:45 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Burke CLA 2005-08-15 19:15:15 EDT
My understanding is that a method with a signature of the form (A< B< T> > p)
should not allow a capture-conversion when passed a parameter of type A< B<?> >
- since the wildcard is not first-level. (Not quite sure If I have the
terminology right here). If it does allow such a conversion, a
ClassCastException may follow.

Here is code that compiles under eclipse 3.1.0 (release), but not under Sun's
javac. I believe the behaviour of Sun's javac is correct - although the JLS
content might be slightly questionable in the area of capture conversion. This
code compiles under eclipse, and when run, throws a ClassCastException (Despite
there being no explicit casts)
<pre>
//This compilation unit is released in the public domain.
package typeBug;

import java.util.ArrayList;
import java.util.List;

/** 
 * This class demonstrates a generic program that Eclipse must not compile as 
 * it can lead to a ClassCastException despite having no explicit type casts.
 */
public class TypeBug
{
   private static class ValueHolder<T>
   {
      public T value;
   }
   
   public static void main(final String[] args)
   {
      List< ValueHolder<?> > multiList = new ArrayList< ValueHolder<?> >();
      
      ValueHolder<Integer> intHolder = new ValueHolder<Integer>();
      intHolder.value = 1;
      
      ValueHolder<Double> doubleHolder = new ValueHolder<Double>();
      doubleHolder.value = 1.5;
      
      multiList.add(intHolder);
      multiList.add(doubleHolder);
      
      // I believe this line is being erroneously treated as a capture
conversion under 3.1 JDT.
      // I believe the problem is that ? cannot be captured except in a first
level wildcard.
      swapFirstTwoValues(multiList);
      
      // this line causes a ClassCastException when checked.
      Integer value = intHolder.value;
      System.out.println(value);
   }
   
   private static <T> void swapFirstTwoValues(List< ValueHolder<T> > multiList)
   {
      ValueHolder<T> intHolder = multiList.get(0);
      ValueHolder<T> doubleHolder = multiList.get(1);
      
      intHolder.value = doubleHolder.value;
   }
}
</pre>

Here is the error message Sun's javac gives:
typeBug/TypeBug.java:33:
<T>swapFirstTwoValues(java.util.List<typeBug.TypeBug.ValueHolder<T>>) in
typeBug.TypeBug cannot be applied to
(java.util.List<typeBug.TypeBug.ValueHolder<?>>)
      swapFirstTwoValues(multiList);
      ^
1 error
Comment 1 Philipe Mulet CLA 2005-08-16 04:33:33 EDT
What makes you think Eclipse did capture anything in this example ?
Comment 2 Philipe Mulet CLA 2005-08-16 04:38:14 EDT
Also, why do you think javac's behavior is correct ? The fact you get a
ClassCastException has nothing to do with this. 

When using wildcards, you bypassed the strong type verifications, and thus
exposed  your code to risk of such exception. Indeed, you defined a
ValueHolder<Integer> but then used it as a ValueHolder<?>, thus loosening the
type rules.

You could have achieved the same effect using raw types. Now, need to double
check to see if this isn't a wildcard invocation scenario; but I suspect our
behavior is just fine.
Comment 3 Daniel Burke CLA 2005-08-16 05:03:49 EDT
Hi Phillipe,

You may be right, I am not sure,
I will double check my terminology and reasoning with other people.

My understanding was that if a Java program compiles without warnings and has no
casts, a ClassCastException should be impossible.
Comment 4 Daniel Burke CLA 2005-08-16 08:12:02 EDT
Hi Phillipe,

please read: bracha.org/wildcards.pdf
This is a paper about the design of Java wildcards and covers this particular case.

Please look at page 4, Wildcard capture - the 6th paragraph.

It explains why A< B< T > > must not be capture-converted for A< B< ? > >

The reason why is because A< B< ? > > must be read as, an A of Bs where the Bs
may have different types (eg. B<Integer>, B<Double>).

Therefore, it is incorrect to capture ? as a T for wildcards that are not
'first-level', since the ? is not one universal types across all Bs contained in
A. (For a much more precise explanation, please read the paper)

In essence, the paper basically explains why wildcards are always statically
type-safe and using them does not compromise the type-safety of the program.
Comment 5 Philipe Mulet CLA 2005-08-16 09:14:58 EDT
I fully agree with what is explained in this article, and JDT only captures top
level wildcards. My question in comment 1 was wondering about why you did think
JDT had captured List<ValueHolder<?>>.

The fact you bypassed typing rules by using wildcards remain though.
Comment 6 Daniel Burke CLA 2005-08-16 09:25:02 EDT
Sorry, to answer your question:

"What makes you think Eclipse did capture anything in this example ?"

mutliList is of type: 
* List< ValueHolder<?> >

swapFirstTwoValues signature is:
* private static <T> void swapFirstTwoValues(List< ValueHolder<T> > multiList)

Therefore, for this line to compile:
swapFirstTwoValues(multiList);

surely, the compiler has captured the ? wildcard as T, no?

Comment 7 Philipe Mulet CLA 2005-08-16 09:53:11 EDT
Yes, T got inferred to be '?' for the generic invocation.
That's not what capturing is about; this is rather standard inference of generic
method type arguments from invocation arguments.

Comment 8 Daniel Burke CLA 2005-08-16 10:13:11 EDT
My understanding may be wrong, but in this case, it seems fairly clear that this
is called 'capture conversion'.

Here is a quote from 3.2 Wildcard captures of the paper linked above:

----
"Wildcards, as described above, turn out to cause a practical
dilemma when defining methods and variables. An example
of this is the static Collections.unmodifiableSet()." 

"A natural signature for this method could be this:
<T> Set<T> unmodifiableSet(Set<T> set) { ... }"

"it is actually safe to allow the
method taking a Set<T> to be called with a Set<?>. We may
not know the actual element type of the Set<?>, but we know
that at the instant when the method is called, the given set
will have some specific element type, and any such element
type would make the invocation typesafe. This mechanism
of allowing a type variable to be instantiated to a wildcard
in some situations is known as wildcard capture, because the
actual run-time type behind the ? is "captured" as T in the
method invocation."
----

I believe this fairly clearly states that the above case with
"swapFirstTwoValues(multiList);" is called "capture".

The paper goes on to say:

----
"Capturing wildcards is only legal in some situations, however.
It must be known that there is a unique type to capture
at runtime. For this reason, a type variable can only capture
one wildcard because, for instance, the actual element
types of two different Set<?>s may be different. Also, only
type variables that occur at "top level" in a generic class (as
in Stack<T> and unlike Stack<Stack<T>> or Stack<T[]>), can
be captured. This is again because two Stack<?> elements of
a Stack<Stack<?>> may have different element types, and so
cannot be captured by the single T in Stack<Stack<T>>."
----

In my example, replace Stack<Stack<T>> with List<ValueHolder<T>>

Supposing it wasn't called 'capture conversion': the point made in the paper
would still apply for this case. The example above is precisely why
capture-conversion must not be allowed for non-"top level" type-parameters.
Comment 9 Philipe Mulet CLA 2005-08-16 17:47:30 EDT
Where did you read my claiming it should be captured ??
Only toplevel wildcards can be captured, as instructed by the spec, and as we allow.

So there is *no capture* involved in there. What happens (again) is standard
generic method type inference (read JLS 3rd edition 15.12.2.7). So unless I am
strongly mistaken, inference is performing on constraints:

List<ValueHolder<?>> << List<ValueHolder<T>>
which yields T = ?
Comment 10 Daniel Burke CLA 2005-08-17 05:35:01 EDT
I'm sorry, you are right, you very clearly stated that eclipse is not capturing
the '?'. The reason why I had assumed wildcard-capture was involved is because I
had assumed there was no other inference. My assumption is based on the
reasoning that if there was another way to infer, wildcard-capture would be
redundant and would not exist.

List<ValueHolder<?>> << List<ValueHolder<T>>
As you say, this constraint does not hold unless inference is used to determine
T = ?
I don't see any such inference.
Comment 11 Daniel Burke CLA 2005-08-17 05:40:04 EDT
Sorry, just to clarify, I don't expect you to prove there is such an inference.
Instead, I will try and show there isn't and make another post here if I do.
(Or I find there is an inference, then I will post that here.)
Comment 12 Philipe Mulet CLA 2005-08-17 09:09:55 EDT
Captures are useful to reduce the power of wildcards when used. It avoids having
type safety issues with wildcards. For instance, two instances of
List<capture-if ?> are not compatible, where non-captured versions List<?> are
compatible to each other. Sometimes captures are a bit overparanoid, but in this
scenario they are not interferring.

Now, I am still wondering about the fact we reject code which javac accepts. I
now suspect once inference has performed, the invoked method has a wildcard
argument which is considered as an unsafe wildcard invocation. If true, this
would indicate we are wrong and javac is right in rejecting this code.
Comment 13 Daniel Burke CLA 2005-08-17 19:21:54 EDT
Hi again,

I've cooked up a proof by contradiction. Please tell me if you see a mistake
which causes my proof to fail.

Assumption, T is inferred as ? using the algorithm from JLS: 15.12.2.2

m = swapFirstTwoValues(List)
e1 = multiList;
A1 = List< ValueHolder<?> >
F1 = List< ValueHolder<T> >
R1 = T
B1 = none (?)

U1 = To determine: using initial constraint List< ValueHolder<?> > << List<
ValueHolder<T> >

Si = To determine: List< ValueHolder<T> >[T = U1]

Lets skip to 15.12.2.7 so we can determine U1.

From the top of the algorithm:

1st recursion: Constraint: A1 << F1 (ie. List< ValueHolder<?> > << List<
ValueHolder<T> >)

Only the 2nd case applies: F1 involves a type parameter T
   Only the 2nd case applies: The constraint has the form A << F
      Only the 4th case applies: F has the form G<U> (G = List, U = ValueHolder<T>)
         The subclause is true: A has a supertype of the form G<V> (V =
ValueHolder<?>)
            We must now recurse with the new constraint V = U (ValueHolder<?> =
ValueHolder<U>)

2nd Recursion: Constraint V = U (ie. ValueHolder<?> = ValueHolder<T>).
V is renamed as A, U is renamed as F.

Only the 2nd case applies: F involves a type parameter T
   Only the 3rd case applies: The constraint has the form A = F

      (Note I had to assume the JLS is faulty here since it says A and F are
letters that are only used for the actual and 

formal parameters, although this isn't quite true as V is not the actual
parameter, nor is U the formal parameter. The 

discussion backs up my assumption)

      Crucial point: Case 1 (F has the form G<U>) does not apply. This is
because V would have to be ?, but V must be a 

type-expression (Wildcards are not considered type expressions by the JLS.)

      The final default case applies instead: Otherwise, no constraint is
implied on Tj. 
         This ends the recursion as no more cases apply.

The 2nd recursion has returned - we are now back in the first recursion. There
are no implied constraints.

Continuing the algorithm: (I am now at the top of page 463 of the pdf version of
the 3.0 JLS)
Since there are no implied constraints, we have to go to: 15.12.2.8

The method is not invoked in an assignment context, therefore, T is inferred as
Object.

T is inferred as Object.
This contradicts T is inferred as ? (Which is written above as T = ?)

The fact that you cannot write:

TypeBug.<?>swapFirstValues(multilist) - backs this up - type inference is just a
conveniance tool - you should be able to explicitly declare the type arguments
if you want. There is in fact no case when T may be inferred as ? which is one
reason why this should not compile.

There is no legal method application here.
Comment 14 Matthew Sackman CLA 2005-08-19 06:52:40 EDT
The english hand-waving arguement for this is the following:

1. Foo<?> is a super type of Foo<T> for all T
2. Method invocation allows widening of the arguement expressions, not narrowing
3. In the code here we have an arguement of type List< ValueHolder<?> > and we
are trying to invoke a method which takes a parameter of type List< ValueHolder<T> >
4. The act of converting List< ValueHolder<?> > to List< ValueHolder<T> > is a
narrowing of the type. This is not permitted by method invocation conversion.
5. The very fact that in one case and one case only it does make sense to allow
the narrowing of a wildcard type parameter '?' to a formal type parameter 'T' is
the reason for wildcard capture. This does not apply here because it this
situation doesn't satisfy the other requirements for wildcard capture (as we all
agree!)
6. Thus the method can't be invoked because doing so is narrowing the type of
the arguement expression which isn't permitted.
Comment 15 Philipe Mulet CLA 2005-08-19 10:51:57 EDT
Thanks to all for pointing out this issue. In particular, I had missed the fact
true wildcards were not elligible for type inference. 
Tuned implementation to enforce this rule, added GenericTypeTest#test806.

Released in 3.1.1 and 3.2 code streams.
Comment 16 Daniel Burke CLA 2005-08-19 18:32:23 EDT
wonderful! thanks
Comment 17 David Audel CLA 2005-09-21 10:39:25 EDT
Verified in I20050921-0010 for 3.2M2
Comment 18 Olivier Thomann CLA 2005-09-26 11:45:46 EDT
Verified for 3.1.1 using M20050923-1430.