Bug 508958 - Windows character limit exceeded during link
Summary: Windows character limit exceeded during link
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: cdt-build-managed (show other bugs)
Version: 8.8.1   Edit
Hardware: PC Windows NT
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-12-09 02:57 EST by Jacek Slimok CLA
Modified: 2020-09-04 15:18 EDT (History)
2 users (show)

See Also:


Attachments
Proposed patch (6.86 KB, patch)
2017-10-30 03:50 EDT, Jacek Slimok CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jacek Slimok CLA 2016-12-09 02:57:29 EST
The builder creates a list of object files to be linked as a single long string to be passed to the linker, together with all the flags. This is a problem for projects with many source files and/or long file paths within the project itself.

Historically this has already been looked at by the build tools makers such as GNU ARM Eclipse build tools. Their attempt to fix the issue was to instead of passing it through command line (with a limit of 8191 or even lower) to pass it through CreateProcess WINAPI instead. This however does not solve the issue, but only raises the character limit to 32767 characters.

This is similar to bug report #72965 however the solution done there is not applicable in this case. From my understanding, in the mentioned report - a problem with clean, as parameter list to rm exceeded character limit - the solution there is to create a new rm command each time a character limit is exceeded.

My suggestion is to pass the list of object file paths through a separate file instead. As far as I'm aware this approach isn't new and it's already done by other IDEs, such as QT Creator and it completely bypasses any limitations. The format of the temporary files looks as follows:
INPUT(
  path/to/file1.o
  path/to/file2.o
  path/to/file3.o
);

Such file is then passed directly as an argument where paths to object files were passed previously.
Comment 1 Jonah Graham CLA 2017-05-27 08:18:53 EDT
Not going to make 9.3. Sorry about the delay.
Comment 2 Jacek Slimok CLA 2017-09-27 09:39:01 EDT
Any ETA on this? We have an in-house solution for this that works just as described with no issues. Having to maintain and distribute a separate version is a bit annoying though.
Comment 3 Jonah Graham CLA 2017-10-13 16:19:25 EDT
(In reply to Jacek Slimok from comment #2)
> Any ETA on this? We have an in-house solution for this that works just as
> described with no issues. Having to maintain and distribute a separate
> version is a bit annoying though.

Can you submit your change please? I have time to review your fix, but I don't know when I will find time to write the fix.
Comment 4 Jacek Slimok CLA 2017-10-30 03:50:53 EDT
Created attachment 271237 [details]
Proposed patch
Comment 5 Jacek Slimok CLA 2017-10-30 04:02:16 EDT
Sorry for late response. I've added the patch file that we use. It fixes both: linking part (where paths to object files were passed through command line parameters) and clean. After the change both are done through files: linking through "@response" file, clean through "sh" given a file with "rm" command for each file to be removed. I'll leave it up to you to decide whether you want to include both fixes. As a sidenote - bigger projects will require both and in our case if only one fix is applied, we'll still need to reintegrate the fix for future releases manually. Therefore from our perspective it would be best to have them both applied.

As for the fix itself, we wanted to keep the number of changed lines to bare minimum, therefore the part where (if useResponseFiles() returns true) we remove the "$(OBJS)" part from output and add the part with response file instead. The optimal solution would be to not add this part at all, but would require more lines of code to be changed and possibly slightly reorganized. I'll also leave this part to you.

The "useResponseFiles" method  returns whether response files should be used - so far always true. If you feel like this should be an option (i.e. project option or global option), this method can probably be used for this purpose.