Bug 258002 - [stats] Statistics for percentage translated are not accurate
Summary: [stats] Statistics for percentage translated are not accurate
Status: RESOLVED FIXED
Alias: None
Product: Babel
Classification: Technology
Component: Server (show other bugs)
Version: unspecified   Edit
Hardware: PC Mac OS X - Carbon (unsup.)
: P2 major (vote)
Target Milestone: ---   Edit
Assignee: Denis Roy CLA
QA Contact:
URL: /babel/index.php
Whiteboard:
Keywords:
Depends on: 254429
Blocks:
  Show dependency tree
 
Reported: 2008-12-08 17:56 EST by Margaret Wong CLA
Modified: 2009-03-31 15:27 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Margaret Wong CLA 2008-12-08 17:56:47 EST
The statistics on this page:
http://babel.eclipse.org/babel/stats.php
Are not accurate.  They include strings that are not translatable (and hence remain in English) and they also include files that are obsolete.  Please change the scripts that calculate these statistics to remove the dependency on the two aforementioned items.
Comment 1 Denis Roy CLA 2008-12-11 10:06:24 EST
Assigning to myself.
Comment 2 Sean Flanigan CLA 2008-12-11 18:41:53 EST
While you're in there, could you do something about statistics for translations which are inherited from related languages (eg en_AU inherits from default locale, and pt_BR inherits from pt)?  

For instance: for text which is identical in en_US and en_AU, en_AU won't need its own translation (with bug 258217, such text might be marked as "use original English").  

And if (enhancement) bug 258216 is implemented, en_AU might inherit from en_GB too.

Do you want this as a separate enhancement request?
Comment 3 Denis Roy CLA 2008-12-11 20:55:35 EST
(In reply to comment #2)
> While you're in there,

[snip]

> Do you want this as a separate enhancement request?


Yes, please.  Can you prefix with [stats] ?   That way It's easier to track all the bugs where the work can be lumped together.

Comment 4 Sean Flanigan CLA 2008-12-14 19:24:13 EST
(In reply to comment #3)
> (In reply to comment #2)
> > Do you want this as a separate enhancement request?
> 
> Yes, please.  Can you prefix with [stats] ?   That way It's easier to track all
> the bugs where the work can be lumped together.

Done.  See bug 258764.
Comment 5 Denis Roy CLA 2009-01-21 16:22:39 EST
After investigation, it appears the stats calculations are accurate.

The problem is that the maps processor isn't removing dead files.

For instance, all (or most) of the files below are dead, yet contain thousands of strings and zero translations.  Once bug 254429 is fixed, that will fix the stats.


|    1087 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.cde.CDE.properties                                                 |           38 |        0 |                                      
|    1088 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.cocoa.Cocoa.properties                                             |           67 |        0 |                                      
|    1089 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.gdip.Gdip.properties                                               |          540 |        0 |                                      
|    1090 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.gnome.GNOME.properties                                             |           40 |        0 |                                      
|    1091 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.gnome.properties                                                   |            1 |        1 |                                      
|    1092 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.gtk.GTK.properties                                                 |           12 |        0 |                                      
|    1093 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.gtk.OS.properties                                                  |         2473 |        0 |                                      
|    1094 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.gtk.properties                                                     |            1 |        1 |                                      
|    1095 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.motif.OS.properties                                                |         1039 |        0 |                                      
|    1096 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.mozilla.XPCOMInit.properties                                       |           71 |        0 |                                      
|    1097 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.ole.win32.COM.properties                                           |          227 |        0 |                                      
|    1098 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.opengl.carbon.AGL.properties                                       |           23 |        0 |                                      
|    1099 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.opengl.glx.GLX.properties                                          |           37 |        0 |                                      
|    1100 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.opengl.win32.WGL.properties                                        |           21 |        0 |                                      
|    1101 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.photon.OS.properties                                               |          525 |        0 |                                      
|    1102 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.properties                                                         |            2 |        2 |                                      
|    1103 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.win32.OS.properties                                                |         1829 |        0 |
|    1104 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.win32.Win32.properties                                             |           59 |        0 |
|    1105 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.opengl.GL.properties                                                        |          160 |        0 |
|    1106 | org.eclipse.swt.tools/JNI Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.opengl.GLU.properties                                                       |           77 |        0 |

Comment 6 Denis Roy CLA 2009-01-23 10:39:37 EST
So the maps processor seems to be removing dead files now.

There remains a problem.  Look at this file:
http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.swt.tools/JNI%20Generation/org/eclipse/swt/tools/internal/org.eclipse.swt.internal.win32.OS.properties?hideattic=0&view=log

This file was removed Wed Sep 10 22:29:20 2008 UTC so it's still a valid file in Eclipse 3.4 (and 3.4.1).  However, it contains thousands of strings that are definitely bogus, non-translatable strings, and these 'untranslated' strings are negatively affecting the stats.

To fix the stats (and to avoid people translating them for nothing), I recommend I simply mark as Inactive all those bogus SWT/JNI_Generation files that are dead today.  Kit, what do you think?
Comment 7 Denis Roy CLA 2009-03-31 14:53:32 EDT
(In reply to comment #5)
> After investigation, it appears the stats calculations are accurate.
> 
> The problem is that the maps processor isn't removing dead files.
> 
> For instance, all (or most) of the files below are dead

A slight precision: the files in my previous comment are dead in the HEAD stream.  For 3.4 and 3.3.1, the files are still 'good', although we know they shouldn't be.

I have marked those files as inactive, and I'll make sure they get excluded from the stats counts accordingly.  I have confirmed that they no longer show up in the UI for translation.
Comment 8 Denis Roy CLA 2009-03-31 15:27:58 EDT
With the files removed and all the strings active, eclipse 3.4 went from 68% complete (for French) to a more realistic 98.7%.

Closing as fixed.  We will probably still get similar "dead" .properties files that should not be in Babel in the future, but we'll just have to mark those as inactive manually.