I agree with need to provide a simple tool-chain integration capability
and to get scanner discovery under control. I want MBS to use this information
rather than requiring duplicate information to be specified and I want MBS to
be able to contribute to the core information when necessary – e.g. paths and
macro definitions users enter into MBS property pages.
So, I’m thinking that tool-chains and builders are different,
and can be mixed and matched to some extent, and this is the simple information
that CDT needs from a tool-chain or builder in order to provide usable, default
settings to new users. Do it seem like too much?
Leo
Tool-chain
The tool-chain element will provide the following metadata:
·
Information describing the tool-chain to the
user – name, supported host platforms, supported target platforms, …
·
Information for populating the CDT new project
AppWizard framework. This will enable the same support as the current MBS and
core functionality do.
·
Information for setting up the build environment
of the process created to run a build
·
Information needed by the CDT editing/code
assistance/refactoring functionality – e.g. the paths and macro definitions
currently provided by scanner discovery and other mechanisms. Note that a
builder (e.g. MBS) may also be involved in providing some of this information.
The tool-chain will be the CDT focal point for this information and may need to
use builder specific mechanisms for gathering information from a builder that
the tool-chain supports.
·
The builder(s) that the tool-chain works with.
There will be a “core builder“ “flavor” that can be used with any tool-chain
that provides the core metadata. Support for additional builders (which require
builder specific metadata) can be specified.
·
The debugger(s) that the tool-chain is known to
work with
·
The libraries that the tool-chain is known to
work with
·
The default error parsers for the tools in the
tool-chain
·
The build console(s) that the tool-chain is
known to work with? I don’t know whether build consoles are associated with a
tool-chain or a builder or are simply a user preference setting.
The “known to work with” metadata will not provide an
exclusive list of other elements that the tool-chain will work with. Users may
try other combinations at their own risk.
Builder
The builder element will provide the following metadata:
·
Information describing the builder to the user –
name, supported host platforms, …
·
The default error parser(s) for the builder
·
Information for setting up the build environment
of the process created for the build
·
Information for creating the command line for a
build
·
Whether it requires additional metadata, and if
so, the name of the extension point
From: cdt-dev-bounces@xxxxxxxxxxx
[mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of James Blackburn
Sent: Tuesday, April 27, 2010 12:37 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Build Again
I definitely feel the pain. We definitely need a scalable
answer to path and symbol persistence in the core - one that isn't tied to any
particular build system. Plugging in an extensible scanner discovery should be
easy for extenders. And scanner discovery would be a first class citizen like a
dwarf contributor, or sdk path contributor - or <your_buildsystem>
contributed paths. Solving this once and for all would sort out what must be
one of the most frequently asked integraor questions on this list.
Moreover it'll allow all the magic stuff the parsers and indexers do now to
work. I believe this is the major cause behind source navigation not working
out the box for users.
There's the common need to fix this and the pain associated.
I think andrew made a fair bit of progress in 6. So soon after 7 lets see how
far we can get if we continue this work and we all chip in.
To Leo's point, have you ever tried to integrate a toolchain
where you don't want MBS, but you do want standard build? The overhead in doing
that and getting (or not getting which is the norm) a working scanner discovery
is what brought me to this point. There is some overlap, but it is actually
quite small. For the android toolchain, it's pretty much the PATH env var and
the names of the compilers and a hook to the gcc scanner discovery.
Unfortunately, I feel we've lost the ability to keep the simple things simple.
It's not clear to me how managed build calls the built-in
discovery stuff. That's probably all part of my confusion with scanner
discovery to begin with. But as I mentioned, I have no intentions of changing
managed build. My suggestion is to reimplement a whole new build stack for
standard build including a clone/reimplementation of scanner discovery, and a
very simple build model.
On Tue, Apr 27, 2010 at 2:27 PM, Chris Recoskie <recoskie@xxxxxxxxxx> wrote:
In our case, the only way we can get at the built-ins for the IBM compiler
is to do a hidden build, similar to what is done for the GNU compilers. There's
no Windows registry to rely upon on AIX or Linux that we could look up such
things in. So we need to have the built-ins discovery run for managed build
projects too. I'm not sure how you'd get around it.
We (IBM) have several compiler versions and languages that we support, on
multiple platforms, and it would be an ugly problem of combinatoric complexity
if we had to hardcode all the different combinations in the toolchain
definitions, or even code it all in Java. It's even worse for the GNU tools.
===========================
Chris Recoskie
Team Lead, IBM CDT and RDT
IBM Toronto
<graycol.gif>Doug Schaefer ---04/27/2010 12:55:59 PM---Thanks, Chris,
I think we're getting a lot of momentum and resourcing to address this. I can't
wait
|
<ecblank.gif>
From:
|
<ecblank.gif>
Doug Schaefer <cdtdoug@xxxxxxxxx>
|
|
<ecblank.gif>
To:
|
<ecblank.gif>
"CDT General developers list." <cdt-dev@xxxxxxxxxxx>
|
|
<ecblank.gif>
Date:
|
<ecblank.gif>
04/27/2010 12:55 PM
|
|
<ecblank.gif>
Subject:
|
<ecblank.gif>
Re: [cdt-dev] Build Again
|
|
<ecblank.gif>
Sent by:
|
<ecblank.gif>
cdt-dev-bounces@xxxxxxxxxxx
|
Thanks, Chris, I think we're getting a lot of
momentum and resourcing to address this. I can't wait to see what we can do.
I'd like to make sure we separate out scanner
discovery from managed build. These are two separate areas that shouldn't
overlap. Again my focus will be on resurrecting a separate standard build
system where scanner discovery is an essential mechanism. Reusing it for
managed build doesn't make sense to me and will probably simplify managed build
if we remove it. For managed build, all you really want is get at the compiler
built-ins and there are probably simpler mechanisms to accomplish that.
:D
On Tue, Apr 27, 2010 at 11:50 AM, James
Blackburn <jamesblackburn@xxxxxxxxx> wrote:
Hi
Chris,
I agree with your summary. I (thankfully) have got rid of scanner discovery
here. The major gripe here is the issue of merging & version
controlling .cproject. The XML model bears little resemblance to a user's
intuitive understanding of the settings set. I've got a .cproject pretty
printer which generates a human readable output from the config, but the fact
that the .cproject changes seemingly randomly and non-obviously, after an
innocuous change (e.g. flipping active configuration) is bad.
I'd like to help out improving this and the scalability issues. I've got some
ideas on that front.
Cheers,
James
On 27 April 2010 16:31, Chris Recoskie <recoskie@xxxxxxxxxx> wrote:
I would have replied to this thread sooner but I
was on vacation yesterday.
Rather than respond to all the different sub-threads, I'll summarize a few
points here.
As I stated recently, I think I can get a significant amount of my time put
towards build system work starting in the summer/fall, with the idea being to
release whatever changes resulted in the June 2011 simultaneous release.
Whether everything I would like to do can actually be accomplished in that time
or not remains to be seen, but it should allow for a good start.
To give some background, we've just GA'ed our first product release based on
our CDT/RDT remote development work... Rational Developer for Power Systems
Software. Basically it's a remote development IDE for the Power platform. Some
issues have come from our customers as a result during our beta program, most
of which surrounded scalability, build, and scanner discovery. As such we've
got the required momentum behind us to make some investment in these areas if
it will address these pain points.
As others have stated, doing a complete rewrite of everything would be ideal in
some ways but I'm not sure it's practical either given the resources. That
being said. we can probably rewrite a few things, and in fact in some cases I
think we'll need to if we want them fixed.
So, here are my main concerns and thoughts. A lot has been said by the rest of
you that I would say I largely agree with... I won't rehash it all, I'll just
add to it.
- Scalability of the build model is a big problem right now. We already had
customers complaining that they have hundreds of projects in the workspace,
which blows up the heap easily. Not only do we need to address the memory
footprint, but the slowness of creating and persisting the C project
description (and particularly the build model portion of it) needs to be
addressed. This slowness gets even slower when you are working with remote
projects, because remote I/O is expensive. I think the C project description
implementation and the persistence mechanisms need a rewrite.
- The elements in the current build model/MBS model themselves are not
necessarily bad as they do a pretty good job of modeling arbitrary toolchain
setups. Those elements can probably be tweaked to help to help address the use
cases where we are still lacking, without throwing the baby out with the
bathwater. I think even if I were to think about redesigning the build model,
I'd probably end up in a similar spot to where it ended up, with a few sins
corrected. So I don't think it's worth it to blow that all away and start
fresh. Most of our ISVs integrate to the build system using the MBS extension
points, so keeping that similar is something I'd consider necessary.
- Autotools support. Some of our bigger customers are using Autotools with the
IBM compilers. Building with them using a standard make project is really
clunky, and scanner discovery doesn't handle the autotools output properly. I
need to look at what the Redhat folks/Linux Tools folks have done to see how
that currently integrates. Does the autotools integration work with any
compiler right now or just GNU? Did they solve the scanner discovery problem? I
don't know.
- Scanner discovery in general is a complete mess and personally I would like
to raze it to the ground and salt the earth upon which it stood so nothing ever
grows there again. I got scanner discovery "sort of" working for our
product but it is rather hacky and doesn't work that great. Per file and per
folder settings don't currently work at all, and our customers care about that
(see: Autotools). The scalability of it sucks as the hidden build to get at the
compiler built-ins still gets invoked way too often for arcane reasons I have
not fully deciphered yet. Part of the problem is the scanner discovery stuff is
entangled in the mess that is the CProjectDescription and its henchmen, so even
if you sort of figure out your scanner discovery parsers and info collectors,
getting the data into the IDE properly is a nightmare.
- Scanner discovery working properly is REALLY important for us. Our customers
have all told us that our big differentiator that makes our product attractive
compared to vi/Emacs and the like is that we have all these great source-aware
features driven by the index. Our customers have really big projects though,
with lots of third party libraries, and they are pretty much unwilling to
configure the scanner info manually. The big customers pretty much all have
existing build processes that they are unwilling to switch away from, which
means none of them will be switching to Managed Build when our product supports
it.. So scanner discovery has to work right.
===========================
Chris Recoskie
Team Lead, IBM CDT and RDT
IBM Toronto
<graycol.gif>Doug
Schaefer ---04/24/2010 11:06:29 AM---Hey gang, FYI, As I mentioned on the last
call, I've been asked to play a more active role
|
<ecblank.gif>
From:
|
<ecblank.gif>
Doug Schaefer <cdtdoug@xxxxxxxxx>
|
|
<ecblank.gif>
To:
|
<ecblank.gif>
"CDT General developers list." <cdt-dev@xxxxxxxxxxx>
|
|
<ecblank.gif>
Date:
|
<ecblank.gif>
04/24/2010 11:06 AM
|
|
<ecblank.gif>
Subject:
|
<ecblank.gif>
[cdt-dev] Build Again
|
|
<ecblank.gif>
Sent by:
|
<ecblank.gif>
cdt-dev-bounces@xxxxxxxxxxx
|
Hey gang, FYI,
As I mentioned on the last call, I've been asked to play a more active role in
TCF and help build a community for it and work with you to bring it to
maturity. That's taking most of my time these days as I get up to speed and
figure out what we need to do with it.
In my spare time, I'm working on bringing the CDT to the Android community to
help build native libraries for that platform. The focus is on one of my dream
projects, building a game engine out of open source parts, in this case for
Android, but I have eyes on iPhone and MeeGo too. Part of doing that is getting
the Android gnu toolchain working well with CDT. And that's brought me back
into the CDT build nightmare. Once again, I am having a hell of a time getting
scanner discovery working for these seemingly simple Makefile projects. Just
look at a stack trace sometime and you'll see, today, the CDT build system is
the opposite of simple.
So I am urging myself to get back into working on a new build system. We need
to put the architecture back to the way it was, with managed build being an
add-on, not having everything driven by managed build. Obviously, I'm not going
to have much time to spend on it, so by definition, I'll have to keep it
simple. And I'll have to do it as a separate stack, not as an evolution of what
we have.
Over the next few days, I'll jot down some more ideas here http://wiki.eclipse.org/CDT/NextGenBuild. And I'll recreate cdt.build.* plug-ins, this time in
a separate folder to avoid confusion. And as always help is appreciated and
hopefully, I'll have new found vested interest in bringing it to completion,
especially since I want to get back working on the Android project.
Cheers,
Doug_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
|
