[wtp-dev] Screen scraping and other changes coming to releng

[David M Williams <david_williams@xxxxxxxxxx>]

After having recovered from the embarrassment
of bug
172361
I've been making some improvements to
our build summary reporting. 

It's not displayed too well yet (and
will get worse before it gets better), 
but the principles are completely changing,
and 
the strategy will be to slowly change
things until complete. 

I've moved us off of _some_ of the platform's
releng tools, so they 
will stop breaking us :) [as they change
their own results pages] and also
so we can do exactly what we want ..
such as capture certain totals
that the platform does not capture ...
such as total number of compile 
logs. Our wtp releng tools will be in
'org.eclipse.wtp.releng.tools' . 

Most important, we need to practice
what we preach and separate 
the web data from the web presentation.
I'll be converting our scripts
to produce only xml files, that "self
document" the build results and summaries, 
and then use some XSL transforms, etc.,
to display these in pleasant and 
meaningful ways. That might even also
include the Phoenix L&F :) 
In theory, the download servers themselves
should not be serving up PHP, 
but instead the normal web servers should
present the pages, and those 
pages use the XML files to point to
the proper data on the download servers. 

Why should you care? 

1) if anyone is (still) scraping the
HTML pages 
to figure out what to automatically
download, then I will be breaking you. 
The good news is, once it's in XML form,
you'll have true data to use, and 
not have to scrape any longer. 

2) if anyone has any ideas .... that
they would like to implement! :) .... then feel 
free to contribute. 

3) If anyone has any questions or comments,
don't hesitate to ask. 

I've opened bug 175972 to use to document
activities in this area. 
https://bugs.eclipse.org/bugs/show_bug.cgi?id=175972