Bug 502090 - [Cocoa] [10.10] Bridge support files should be updated to latest macOS supported version
Summary: [Cocoa] [10.10] Bridge support files should be updated to latest macOS suppor...
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.6   Edit
Hardware: PC Mac OS X
: P3 normal (vote)
Target Milestone: 4.7 M7   Edit
Assignee: Arun Thondapu CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 506092
Blocks: 497946 502711
  Show dependency tree
 
Reported: 2016-09-24 04:08 EDT by Mikaël Barbero CLA
Modified: 2017-03-24 13:48 EDT (History)
7 users (show)

See Also:


Attachments
gen_bridge_metadata log with tons of errors (1.95 MB, text/plain)
2016-10-14 08:08 EDT, Gunnar Wagenknecht CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mikaël Barbero CLA 2016-09-24 04:08:35 EDT
Copy paste of the original question from https://dev.eclipse.org/mhonarc/lists/platform-swt-dev/msg08035.html

It seems that the bridge support files for the Cocoa port are quite outdated. From https://www.eclipse.org/eclipse/development/plans/eclipse_project_plan_4_7.xml#target_environments, the minimum version of supported macOS is 10.11, but there lack some classes/protocols/... from the bridge support files for this version. 

I tried to copy and override the bridgesupport from my own system:

cp /System/Library/Frameworks/AppKit.framework/Resources/BridgeSupport/AppKit.bridgesupport \
path/to/git/eclipse.platform.swt/bundles/org.eclipse.swt/Eclipse\ SWT\ PI/cocoa/org/eclipse/swt/internal/cocoa/FoundationFull.bridgesupport

and it works well (at least, missing classes/protocols/.../ show up correctly in the Mac Gen view), but when I try to generate again, it fails with NPE: 

java.lang.NullPointerException
at org.eclipse.swt.tools.internal.MacGenerator.getCType(MacGenerator.java:1450)
at org.eclipse.swt.tools.internal.MacGenerator.generateCustomCallbacks(MacGenerator.java:1509)
at org.eclipse.swt.tools.internal.MacGenerator.generateMainClass(MacGenerator.java:790)
at org.eclipse.swt.tools.internal.MacGenerator.generate(MacGenerator.java:191)
at org.eclipse.swt.tools.internal.MacGeneratorUI.generate(MacGeneratorUI.java:320)
at org.eclipse.swt.tools.views.MacGeneratorView$GenJob.run(MacGeneratorView.java:40)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

It actually fails logically, as a comparison between the current HEAD and my own bridgesupport file shows that declared_type is not specified in my system files. 

----

Fixing this bug should also be the occasion to document how to update the bridge support file ;)
Comment 1 Till Brychcy CLA 2016-09-24 15:17:32 EDT
When you do that, can please make sure that the binding for NSView scrollRect:by: is included?

https://developer.apple.com/reference/appkit/nsview/1483497-scrollrect?language=objc

I think this will be useful to replace NSCopyBits, which is deprecated.
https://developer.apple.com/reference/appkit/1473630-nscopybits
Comment 2 Markus Keller CLA 2016-09-26 06:53:39 EDT
(In reply to Mikaël Barbero from comment #0)
> From
> https://www.eclipse.org/eclipse/development/plans/eclipse_project_plan_4_7.xml#target_environments,
> the minimum version of supported macOS is 10.11, [..]

That's not what the plan says. OS X 10.11 is the reference platform, i.e. the configuration that is most thoroughly tested.

The minimal platform for SWT is usually the oldest platform that is practical to be supported by the SWT team, or the oldest that is officially supported by the OS provider. AFAIK, Apple doesn't publish EOL dates for OS X, but from recent security updates, we can assume that at least 10.9 is still "alive". So unless there are technical reasons, we should still aim for 10.9 compatibility.
Comment 3 Mikaël Barbero CLA 2016-09-26 08:48:05 EDT
(In reply to Markus Keller from comment #2)
> That's not what the plan says. OS X 10.11 is the reference platform, i.e.
> the configuration that is most thoroughly tested.

Agreed, I've overlooked the definition. Thanks for the heads-up.

> AFAIK, Apple doesn't publish EOL dates for OS
> X, but from recent security updates, we can assume that at least 10.9 is
> still "alive". So unless there are technical reasons, we should still aim
> for 10.9 compatibility.

Apple usually releases security updates for current, n-1 and n-2 releases. With the recent release of 10.12 (Sierra), that would make 10.10 (Yosemite) the minimum target. WDYT?
Comment 4 Marc-André Laperle CLA 2016-09-26 09:55:47 EDT
(In reply to Mikaël Barbero from comment #3)
> (In reply to Markus Keller from comment #2)
> > That's not what the plan says. OS X 10.11 is the reference platform, i.e.
> > the configuration that is most thoroughly tested.

The tests on Hudson still run on 10.10 though :). See bug 478954.
 
> > AFAIK, Apple doesn't publish EOL dates for OS
> > X, but from recent security updates, we can assume that at least 10.9 is
> > still "alive". So unless there are technical reasons, we should still aim
> > for 10.9 compatibility.
> 
> Apple usually releases security updates for current, n-1 and n-2 releases.
> With the recent release of 10.12 (Sierra), that would make 10.10 (Yosemite)
> the minimum target. WDYT?

+1. I think that some people are staying on 10.9 because of the big UI change in 10.10 but I don't know what that proportion is. If I look at the Steam HW Survey (which might not be representative at all of the Eclipse user base!), 10.9 sits at 6% of macOS users. At least, if 10.9 stays supported in Neon.x, I think it would be reasonable for Oxygen to require 10.10 as minimum.
Comment 5 Arun Thondapu CLA 2016-10-12 11:20:07 EDT
I was able to fish out the command (gen_bridge_metadata) that was used to generate the current bridge support files on 10.7 but that doesn't seem to work on 10.10, or at least I haven't gotten it to work yet, may be the arguments/flags we're using need to be tweaked a little bit.

I'll try once again and provide an update in a day or two...
Comment 6 Mikaël Barbero CLA 2016-10-12 11:44:09 EDT
(In reply to Arun Thondapu from comment #5)
> I was able to fish out the command (gen_bridge_metadata) that was used to
> generate the current bridge support files on 10.7 but that doesn't seem to
> work on 10.10, or at least I haven't gotten it to work yet, may be the
> arguments/flags we're using need to be tweaked a little bit.
> 
> I'll try once again and provide an update in a day or two...

Where are the sources of the command gen_bridge_metadata?
Comment 7 Gunnar Wagenknecht CLA 2016-10-14 05:46:50 EDT
(In reply to Mikaël Barbero from comment #6)
> (In reply to Arun Thondapu from comment #5)
> > I was able to fish out the command (gen_bridge_metadata) that was used to
> > generate the current bridge support files on 10.7 but that doesn't seem to
> > work on 10.10, or at least I haven't gotten it to work yet, may be the
> > arguments/flags we're using need to be tweaked a little bit.

I seem to get good results with:
gen_bridge_metadata -d -F complete --framework AppKit.framework -o AppKitFull.bridgesupport

> Where are the sources of the command gen_bridge_metadata?

That's an Apple utility that comes with xcode/osx.
Comment 8 Mikaël Barbero CLA 2016-10-14 07:43:42 EDT
(In reply to Gunnar Wagenknecht from comment #7)
> (In reply to Mikaël Barbero from comment #6)
> > (In reply to Arun Thondapu from comment #5)
> > > I was able to fish out the command (gen_bridge_metadata) that was used to
> > > generate the current bridge support files on 10.7 but that doesn't seem to
> > > work on 10.10, or at least I haven't gotten it to work yet, may be the
> > > arguments/flags we're using need to be tweaked a little bit.
> 
> I seem to get good results with:
> gen_bridge_metadata -d -F complete --framework AppKit.framework -o
> AppKitFull.bridgesupport

Terrific, thanks Gunnar!

> 
> > Where are the sources of the command gen_bridge_metadata?
> 
> That's an Apple utility that comes with xcode/osx.

Thanks and sorry for the misunderstanding.
Comment 9 Gunnar Wagenknecht CLA 2016-10-14 08:08:51 EDT
Created attachment 264853 [details]
gen_bridge_metadata log with tons of errors

I digged deeper.

While I was able to generate the bridgesupport files. The MacGenerator fails on them with the following stack strace:

java.lang.NullPointerException
at org.eclipse.swt.tools.internal.MacGenerator.generateFunctions(MacGenerator.java:1900)
at org.eclipse.swt.tools.internal.MacGenerator.generateMainClass(MacGenerator.java:815)
at org.eclipse.swt.tools.internal.MacGenerator.generate(MacGenerator.java:191)
at org.eclipse.swt.tools.internal.MacGeneratorUI.generate(MacGeneratorUI.java:320)
at org.eclipse.swt.tools.internal.MacGeneratorUI.lambda$6(MacGeneratorUI.java:315)

I've pushed a review to have a more helpful error message.
https://git.eclipse.org/r/83205

In this particular case, the function 'NSSearchPathForDirectoriesInDomains' is missing in the new 'FoundationFull.bridgesupport'. It's still in FoundationFull.bridgesupport.extra. That's why the generator wants to generate code for it. The function is actually called. Thus, it is required. It's also not deprecated. Thus, it should be there.

I noticed that during generation a ton of errors are thrown. This doesn't seem right. I've attached the log output for people on this bug. Maybe they have an idea.
Comment 10 Eclipse Genie CLA 2016-10-14 12:52:22 EDT
New Gerrit change created: https://git.eclipse.org/r/83254
Comment 11 Eclipse Genie CLA 2016-10-14 12:52:24 EDT
New Gerrit change created: https://git.eclipse.org/r/83253
Comment 12 Eclipse Genie CLA 2016-10-14 12:52:26 EDT
New Gerrit change created: https://git.eclipse.org/r/83255
Comment 13 Gunnar Wagenknecht CLA 2016-10-14 12:56:35 EDT
I was able to make *a lot* more progress. I'm now closer to have MacGenerator generate something that compiles. Currently it does not because of a few new language features in objective c. You can follow the reviews for details. :)

Current compile errors still open:
- instancetype (this is tricky as we actually need to interpret it)
- few more missing types (ObjectType, KeyType)
- few missing fields/methods


In general, this is progress. There was quite some change in the BridgeSupport files.
Comment 14 Gunnar Wagenknecht CLA 2016-10-14 12:58:30 EDT
I've pushed a backup of my work (for the curious follower) here:
https://git.eclipse.org/r/83256
Comment 15 Eclipse Genie CLA 2016-10-15 03:21:26 EDT
New Gerrit change created: https://git.eclipse.org/r/83290
Comment 16 Eclipse Genie CLA 2016-10-15 03:21:33 EDT
New Gerrit change created: https://git.eclipse.org/r/83289
Comment 17 Eclipse Genie CLA 2016-10-15 03:21:35 EDT
New Gerrit change created: https://git.eclipse.org/r/83291
Comment 18 Mikaël Barbero CLA 2016-10-17 14:49:08 EDT
Awesome work Gunnar. See https://dev.eclipse.org/mhonarc/lists/platform-swt-dev/msg07731.html and https://git.eclipse.org/r/#/c/44460/ about the issues I've already raised about XULRunner.
Comment 19 Eclipse Genie CLA 2016-10-18 03:37:45 EDT
New Gerrit change created: https://git.eclipse.org/r/83419
Comment 20 Eclipse Genie CLA 2016-10-18 03:37:57 EDT
New Gerrit change created: https://git.eclipse.org/r/83417
Comment 21 Arun Thondapu CLA 2016-10-20 15:47:56 EDT
Sorry I haven't been able to get to this one yet as I got pulled into a couple of other issues that needed to be looked at this week, will hopefully be able to find some time tomorrow...
Comment 22 Arun Thondapu CLA 2016-10-26 07:09:23 EDT
Hi Gunnar,

I've started reviewing the patches again and should be done by tomorrow but that will be too late for M3, moving to M4, we can plan to merge the patches after M3 is declared later this week.
Comment 23 Mikaël Barbero CLA 2016-11-03 09:06:41 EDT
I'm currently re-installing Yosemite (10.10) to re-generate the bridge files. Stay tuned
Comment 24 Eclipse Genie CLA 2016-11-04 12:30:28 EDT
New Gerrit change created: https://git.eclipse.org/r/84484
Comment 25 Mikaël Barbero CLA 2016-11-04 12:32:06 EDT
Eventually managed to do it. Here are the bridge files for 10.10 https://git.eclipse.org/r/84484. Did not run any diff with Gunnar's files. May be worth it to see the differences between 10.11 and 10.10.
Comment 26 Arun Thondapu CLA 2016-11-08 11:05:12 EST
Gunnar and Mikael, Sorry the reviews got delayed once again as I was on vacation recently, I plan to re-look at the patches tomorrow.
Comment 27 Mikaël Barbero CLA 2016-11-23 03:11:11 EST
Arun, do you have any feedbacks?
Comment 28 Mikaël Barbero CLA 2017-01-04 08:12:05 EST
Arun, do you have any feedbacks?
Comment 29 Arun Thondapu CLA 2017-01-04 10:57:47 EST
(In reply to Mikaël Barbero from comment #28)
> Arun, do you have any feedbacks?

Hi Mikael,

I had to confirm whether we need a CQ for this contribution because of the usage of the gen_bridge_metadata utility from Mac OS X, and it looks like we don't need one (confirmed with Silenio who had generated the files for the first time with help from Apple engineers).

I've started reviewing the patches once again and should have more updates in a day or two. Thanks!
Comment 30 Arun Thondapu CLA 2017-01-24 04:48:58 EST
(In reply to Arun Thondapu from comment #29)
> I've started reviewing the patches once again and should have more updates
> in a day or two. Thanks!

Sorry for the delays once again, there were a few other distractions and it took a bit longer time than anticipated to complete the reviews. I have now gone through all the changes in the patches and did some testing, the gerrit patches look good to be merged but unfortunately we're in the code freeze week for M5 and cannot release the changes now. Moving to M6, will merge the changes right after M5 is declared.
Comment 31 Gunnar Wagenknecht CLA 2017-01-24 06:23:17 EST
Cool. Thanks for the update!
Comment 33 Arun Thondapu CLA 2017-02-07 13:45:29 EST
I've merged the updated bridge support files from 10.10. I will generate the source changes and merge them tomorrow.
Comment 34 Arun Thondapu CLA 2017-03-07 04:25:11 EST
(In reply to Arun Thondapu from comment #33)
> I've merged the updated bridge support files from 10.10. I will generate the
> source changes and merge them tomorrow.

I was going through Gunnar's patches which contain the necessary source and Mac Generator changes but there were minor modifications needed as those patches are based on bridge support files from 10.11 but we need to use the ones from 10.10 for now. I will push the patches with the needed minor tweaks but its again the testing week for M6 and so cannot do it now, moving to M7 for completing the remaining work on this...
Comment 38 Eclipse Genie CLA 2017-03-23 07:32:54 EDT
New Gerrit change created: https://git.eclipse.org/r/93706
Comment 44 Arun Thondapu CLA 2017-03-24 12:56:30 EDT
(In reply to Eclipse Genie from comment #43)
> Gerrit change https://git.eclipse.org/r/83254 was merged to [master].
> Commit:
> http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/
> ?id=70ea713017537c5b5f00e7c46e8e473978a96f58

This patch is not exactly necessary right now as it deals with null annotation information that is present in the bridge support files from macOS 10.11. I merged it nevertheless, as it would help when we do the next round of updates to the bridge support files from 10.10 to 10.11.
Comment 45 Arun Thondapu CLA 2017-03-24 13:38:24 EDT
I think we can consider this done now with respect to the updates corresponding to macOS 10.10. I'll raise a follow up bug for updating the bridge support files to macOS 10.11 in the 4.8 release.