[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [cdt-dev] Build Again

It creates some of the same old problems if we resurrect a separate standard build system that does not interact with the managed build system.  This is why I would prefer to see a “core build system” that all tool-chain integrations use, and then builder specific “extensions” for builders that need it.  The “core build system” would implement the current “scanner discovery” functionality, by integrating all of the possible sources of “paths” and “macro definitions” and being the single provider of this information to the rest of CDT.

 

One reason is that as a tool-chain provider I may want to support multiple builders – for example both “standard make” and MBS.  When my tool-chain is used with MBS, I don’t need or want the scanner discovery mechanism to be invoked.  When a user decides to use their own make file with my tool-chain, then I do need to run a “scanner discovery” mechanism that my tool-chain defines.   This creates problems today and will still create problems with a separate standard make vs. MBS.

 

Leo

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Doug Schaefer
Sent: Tuesday, April 27, 2010 9:55 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Build Again

 

Thanks, Chris, I think we're getting a lot of momentum and resourcing to address this. I can't wait to see what we can do.

 

I'd like to make sure we separate out scanner discovery from managed build. These are two separate areas that shouldn't overlap. Again my focus will be on resurrecting a separate standard build system where scanner discovery is an essential mechanism. Reusing it for managed build doesn't make sense to me and will probably simplify managed build if we remove it. For managed build, all you really want is get at the compiler built-ins and there are probably simpler mechanisms to accomplish that.

 

:D

 

On Tue, Apr 27, 2010 at 11:50 AM, James Blackburn <jamesblackburn@xxxxxxxxx> wrote:

Hi Chris,

I agree with your summary. I (thankfully) have got rid of scanner discovery here.  The major gripe here is the issue of merging & version controlling .cproject.  The XML model bears little resemblance to a user's intuitive understanding of the settings set. I've got a .cproject pretty printer which generates a human readable output from the config, but the fact that the .cproject changes seemingly randomly and non-obviously, after an innocuous change (e.g. flipping active configuration) is bad.

I'd like to help out improving this and the scalability issues. I've got some ideas on that front.

Cheers,
James

 

On 27 April 2010 16:31, Chris Recoskie <recoskie@xxxxxxxxxx> wrote:

I would have replied to this thread sooner but I was on vacation yesterday.

Rather than respond to all the different sub-threads, I'll summarize a few points here.

As I stated recently, I think I can get a significant amount of my time put towards build system work starting in the summer/fall, with the idea being to release whatever changes resulted in the June 2011 simultaneous release. Whether everything I would like to do can actually be accomplished in that time or not remains to be seen, but it should allow for a good start.

To give some background, we've just GA'ed our first product release based on our CDT/RDT remote development work... Rational Developer for Power Systems Software. Basically it's a remote development IDE for the Power platform. Some issues have come from our customers as a result during our beta program, most of which surrounded scalability, build, and scanner discovery. As such we've got the required momentum behind us to make some investment in these areas if it will address these pain points.

As others have stated, doing a complete rewrite of everything would be ideal in some ways but I'm not sure it's practical either given the resources. That being said. we can probably rewrite a few things, and in fact in some cases I think we'll need to if we want them fixed.

So, here are my main concerns and thoughts. A lot has been said by the rest of you that I would say I largely agree with... I won't rehash it all, I'll just add to it.

- Scalability of the build model is a big problem right now. We already had customers complaining that they have hundreds of projects in the workspace, which blows up the heap easily. Not only do we need to address the memory footprint, but the slowness of creating and persisting the C project description (and particularly the build model portion of it) needs to be addressed. This slowness gets even slower when you are working with remote projects, because remote I/O is expensive. I think the C project description implementation and the persistence mechanisms need a rewrite.

- The elements in the current build model/MBS model themselves are not necessarily bad as they do a pretty good job of modeling arbitrary toolchain setups. Those elements can probably be tweaked to help to help address the use cases where we are still lacking, without throwing the baby out with the bathwater. I think even if I were to think about redesigning the build model, I'd probably end up in a similar spot to where it ended up, with a few sins corrected. So I don't think it's worth it to blow that all away and start fresh. Most of our ISVs integrate to the build system using the MBS extension points, so keeping that similar is something I'd consider necessary.

- Autotools support. Some of our bigger customers are using Autotools with the IBM compilers. Building with them using a standard make project is really clunky, and scanner discovery doesn't handle the autotools output properly. I need to look at what the Redhat folks/Linux Tools folks have done to see how that currently integrates. Does the autotools integration work with any compiler right now or just GNU? Did they solve the scanner discovery problem? I don't know.

- Scanner discovery in general is a complete mess and personally I would like to raze it to the ground and salt the earth upon which it stood so nothing ever grows there again. I got scanner discovery "sort of" working for our product but it is rather hacky and doesn't work that great. Per file and per folder settings don't currently work at all, and our customers care about that (see: Autotools). The scalability of it sucks as the hidden build to get at the compiler built-ins still gets invoked way too often for arcane reasons I have not fully deciphered yet. Part of the problem is the scanner discovery stuff is entangled in the mess that is the CProjectDescription and its henchmen, so even if you sort of figure out your scanner discovery parsers and info collectors, getting the data into the IDE properly is a nightmare.

- Scanner discovery working properly is REALLY important for us. Our customers have all told us that our big differentiator that makes our product attractive compared to vi/Emacs and the like is that we have all these great source-aware features driven by the index. Our customers have really big projects though, with lots of third party libraries, and they are pretty much unwilling to configure the scanner info manually. The big customers pretty much all have existing build processes that they are unwilling to switch away from, which means none of them will be switching to Managed Build when our product supports it.. So scanner discovery has to work right.

===========================
Chris Recoskie
Team Lead, IBM CDT and RDT
IBM Toronto

Inactive hide details for Doug Schaefer ---04/24/2010 11:06:29 AM---Hey gang, FYI, As I mentioned on the last call, I've been aDoug Schaefer ---04/24/2010 11:06:29 AM---Hey gang, FYI, As I mentioned on the last call, I've been asked to play a more active role


From:


Doug Schaefer <cdtdoug@xxxxxxxxx>


To:


"CDT General developers list." <cdt-dev@xxxxxxxxxxx>


Date:


04/24/2010 11:06 AM


Subject:


[cdt-dev] Build Again


Sent by:


cdt-dev-bounces@xxxxxxxxxxx





Hey gang, FYI,

As I mentioned on the last call, I've been asked to play a more active role in TCF and help build a community for it and work with you to bring it to maturity. That's taking most of my time these days as I get up to speed and figure out what we need to do with it.

In my spare time, I'm working on bringing the CDT to the Android community to help build native libraries for that platform. The focus is on one of my dream projects, building a game engine out of open source parts, in this case for Android, but I have eyes on iPhone and MeeGo too. Part of doing that is getting the Android gnu toolchain working well with CDT. And that's brought me back into the CDT build nightmare. Once again, I am having a hell of a time getting scanner discovery working for these seemingly simple Makefile projects. Just look at a stack trace sometime and you'll see, today, the CDT build system is the opposite of simple.

So I am urging myself to get back into working on a new build system. We need to put the architecture back to the way it was, with managed build being an add-on, not having everything driven by managed build. Obviously, I'm not going to have much time to spend on it, so by definition, I'll have to keep it simple. And I'll have to do it as a separate stack, not as an evolution of what we have.

Over the next few days, I'll jot down some more ideas here http://wiki.eclipse.org/CDT/NextGenBuild. And I'll recreate cdt.build.* plug-ins, this time in a separate folder to avoid confusion. And as always help is appreciated and hopefully, I'll have new found vested interest in bringing it to completion, especially since I want to get back working on the Android project.

Cheers,

Doug_______________________________________________


cdt-dev mailing list
cdt-dev@xxxxxxxxxxx


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 

GIF image

PNG image

PNG image