Bug 441568 (Webkit2Port4.7) - [GTK3][webkit] Port SWT Browser to webkit2gtk (4.7)
Summary: [GTK3][webkit] Port SWT Browser to webkit2gtk (4.7)
Status: RESOLVED FIXED
Alias: Webkit2Port4.7
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.5   Edit
Hardware: PC Linux-GTK
: P3 major with 1 vote (vote)
Target Milestone: 4.7   Edit
Assignee: Leo Ufimtsev CLA
QA Contact:
URL:
Whiteboard:
Keywords: plan
Depends on: 425123 WebkitJsExecSami webkitFocusListener 497122 499387 505418 508217 509615 510183 510776 Webkit2FuncRetVal 510972 511228 511797 511849 512001 513492 513837 Webkit2_gettext webkit2NotRefWebkit1
Blocks: webkit2Port4.8
  Show dependency tree
 
Reported: 2014-08-12 05:41 EDT by Arun Thondapu CLA
Modified: 2017-11-29 18:03 EST (History)
17 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Arun Thondapu CLA 2014-08-12 05:41:22 EDT
+++ This bug was initially created as a clone of Bug #425123 +++

Bug 425123 provided an initial implementation of the Webkit2 port for the Browser widget on GTK+ 3.  This bug is to complete the implementation of the Webkit2 port during the Mars release.
Comment 1 Sami Wagiaalla CLA 2014-08-12 10:01:52 EDT
I have a couple of patches pending review which complete the port:
https://git.eclipse.org/r/#/c/25157/
https://git.eclipse.org/r/#/c/23416/

There are a few issues however that I am still working around.
Comment 2 Arun Thondapu CLA 2015-04-16 03:09:57 EDT
Deferring this item to 4.6 as it has not seen much activity till now and I don't think it can be completed before M7 which is the feature freeze. It should not adversely impact anyone though, as webkit2 is still not used by default on Linux, it is only used when explicitly enabled by the SWT_WEBKIT2 environment variable.
Comment 3 Arun Thondapu CLA 2015-12-15 02:03:53 EST
Deferring once again as its unlikely this will get done in time for 4.6 as per the current cycles, its still desirable to herd this to completion but may not happen realistically in time for 4.6 as there are other items of priority on Linux currently...
Comment 4 Sergey Nazarov CLA 2016-02-08 05:03:56 EST
Additionally function browser.setUrl(url, postData, headers) not working under GTK+ 3 and webkit2gtk
Comment 5 Ned Twigg CLA 2016-06-29 16:20:08 EDT
To date, we have not succeeded in playing whack-a-mole with every native browser platform, and it's very difficult for app developers to test against the quirks of each browser on each platform.

FWIW, I'd rather declare backruptcy on supporting every native browser, and focus on doing a good Chromium embed.

Look at electron's adoption #'s:
http://electron.atom.io/blog/2016/05/11/electron-1-0

There's a lot of people trying to build native apps with HTML front-ends.  IF you use Electron, you only have one browser to test against.  If you use SWT, you might not even have a working browser at all.

I wish that some of these: http://electron.atom.io/apps/ were in our community, helping us fix our bugs :)
Comment 6 Patrik Suzzi CLA 2016-06-30 05:56:16 EDT
(In reply to Ned Twigg from comment #5)
> To date, we have not succeeded in playing whack-a-mole with every native
> browser platform, and it's very difficult for app developers to test against
> the quirks of each browser on each platform.
> 
> FWIW, I'd rather declare backruptcy on supporting every native browser, and
> focus on doing a good Chromium embed.

We still need to support our users. Several libwebkitgtk / webkit2 issues are perceived as Eclipse / GTK issues. This webkit bug is a pain. I think this should have high priority.

Embedding Chromium is feasible as it is MIT-licensed and we're going toward a more flexible IP management system (#1). I think this should be done after we fixed the main libwebkitgtk / webkit2 issues.

> Look at electron's adoption #'s:
> http://electron.atom.io/blog/2016/05/11/electron-1-0
> 
> There's a lot of people trying to build native apps with HTML front-ends. 
> IF you use Electron, you only have one browser to test against.  If you use
> SWT, you might not even have a working browser at all.
> 
> I wish that some of these: http://electron.atom.io/apps/ were in our
> community, helping us fix our bugs :)

Well, the Atom guys are doing good at marketing (i.e. #2), the product is pretty decent (snappy as eclipse), and they get contributions from javascript developers. 

JS is the most used language nowadays, and if you open Eclipse to JavaScript contributions, in few months you'll start having a good number of new projects. 

Please, check this with JSDT guys (#3). the JSDT Project is already embedding Chromium in Eclipse, to use it as JS debugger. 

#1 https://mmilinkov.wordpress.com/2016/06/29/overhauling-ip-management-at-the-eclipse-foundation/
#2 https://www.youtube.com/watch?v=Y7aEiVwBAdk
#3 https://projects.eclipse.org/projects/webtools.jsdt/who
Comment 7 Alexander Kurtakov CLA 2016-06-30 06:15:00 EDT
(In reply to Patrik Suzzi from comment #6)
> (In reply to Ned Twigg from comment #5)
> > To date, we have not succeeded in playing whack-a-mole with every native
> > browser platform, and it's very difficult for app developers to test against
> > the quirks of each browser on each platform.
> > 
> > FWIW, I'd rather declare backruptcy on supporting every native browser, and
> > focus on doing a good Chromium embed.
> 
> We still need to support our users. Several libwebkitgtk / webkit2 issues
> are perceived as Eclipse / GTK issues. This webkit bug is a pain. I think
> this should have high priority.
> 
> Embedding Chromium is feasible as it is MIT-licensed and we're going toward
> a more flexible IP management system (#1). I think this should be done after
> we fixed the main libwebkitgtk / webkit2 issues.

Somehow I don't envision embedding Chromium feasible solution. Reasons for that are as follow:
* Chromium supports way less os/archs than SWT does AFAIK. I seriously doubt it would support solaris/sparc, linux/ppc64le, aix, hpux and etc. 
* SWT would have to build and distribute these chromium binaries with itself (as they are not available preinsalled on any OS (except for Chromium OS probably)). This opens the Pandora box of building on ancient systems in a desperate hope for maximum compatibility achieved. 
If you combine the two the situation gets in no way better than the webkit one. With webkit at least it's available pre-built from the linux distros and installed by default (except for cases like this one where due to insufficient manpower we fail to keep the pace with changes). 
Don't get me wrong - if someone wants to work on support for Chromium he/she/they is/are more than welcome. But it is not in a position to be the universal solution for all SWT users. Webkitgtk being the only one getting at least remotely close to that is why I would put my team resources behind it (when available).


> 
> > Look at electron's adoption #'s:
> > http://electron.atom.io/blog/2016/05/11/electron-1-0
> > 
> > There's a lot of people trying to build native apps with HTML front-ends. 
> > IF you use Electron, you only have one browser to test against.  If you use
> > SWT, you might not even have a working browser at all.
> > 
> > I wish that some of these: http://electron.atom.io/apps/ were in our
> > community, helping us fix our bugs :)
> 
> Well, the Atom guys are doing good at marketing (i.e. #2), the product is
> pretty decent (snappy as eclipse), and they get contributions from
> javascript developers. 
> 
> JS is the most used language nowadays, and if you open Eclipse to JavaScript
> contributions, in few months you'll start having a good number of new
> projects. 
> 
> Please, check this with JSDT guys (#3). the JSDT Project is already
> embedding Chromium in Eclipse, to use it as JS debugger. 
> 
> #1
> https://mmilinkov.wordpress.com/2016/06/29/overhauling-ip-management-at-the-
> eclipse-foundation/
> #2 https://www.youtube.com/watch?v=Y7aEiVwBAdk
> #3 https://projects.eclipse.org/projects/webtools.jsdt/who
Comment 8 Leo Ufimtsev CLA 2016-06-30 11:35:34 EDT
(In reply to Sami Wagiaalla from comment #1)
> I have a couple of patches pending review which complete the port:
> https://git.eclipse.org/r/#/c/25157/
> https://git.eclipse.org/r/#/c/23416/
> 
> There are a few issues however that I am still working around.

Let me look into this business and continue where Sami left off.
Comment 9 Ned Twigg CLA 2016-06-30 12:46:07 EDT
> Chromium supports way less os/archs than SWT does AFAIK. I seriously doubt it would support solaris/sparc, linux/ppc64le, aix, hpux and etc.

This is *exactly* the "Innovator's Dilemma" which has brought down so many great companies.  Digital photography wasn't as good as film, so Kodak didn't pursue it until it was too far behind, so Kodak is gone.  Many other examples.

IMO, the Electron framework is in the process of throughly wallopping Eclipse RCP.  A huge number of new dev (and non-dev) tools have been built on it already, and it's only two years old.

http://electron.atom.io/apps/

What this shows is that there is a large market of developers who want to build native desktop apps with a latest-and-greatest HTML5 interface.  A lot of java devs want to be able to do this too.  Right now their best bet is to learn Node or some other compile-to-JS language.  If we had a great Chromium embed, their best bet would be Eclipse RCP.

Instead of learning from our competition, we're playing whackamole on platforms with <1% marketshare and implementing CSS on top of Windows MFC widgets.
Comment 10 Leo Ufimtsev CLA 2016-06-30 16:09:00 EDT
(In reply to Ned Twigg from comment #9)
> > Chromium supports way less os/archs than SWT does AFAIK. I seriously doubt it would support solaris/sparc, linux/ppc64le, aix, hpux and etc.
> 
> This is *exactly* the "Innovator's Dilemma" which has brought down so many
> great companies.  Digital photography wasn't as good as film, so Kodak
> didn't pursue it until it was too far behind, so Kodak is gone.  Many other
> examples.
> 
> IMO, the Electron framework is in the process of throughly wallopping
> Eclipse RCP.  A huge number of new dev (and non-dev) tools have been built
> on it already, and it's only two years old.
> 
> http://electron.atom.io/apps/
> 
> What this shows is that there is a large market of developers who want to
> build native desktop apps with a latest-and-greatest HTML5 interface.  A lot
> of java devs want to be able to do this too.  Right now their best bet is to
> learn Node or some other compile-to-JS language.  If we had a great Chromium
> embed, their best bet would be Eclipse RCP.
> 
> Instead of learning from our competition, we're playing whackamole on
> platforms with <1% marketshare and implementing CSS on top of Windows MFC
> widgets.

Linux SWT/Gtk3 pairs nicely with webkitgtk. Gnome browsers such as (Web) Epiphany and Midori use it quite successfully. I don't see anything wrong with using webkitgtk. Electron doesn't seem to be a good fit for SWT/Gtk3.

If someone is interested, it's possible to create Electron as a separate style in SWT, ex new Browser(shell, STW.ELECTRON) and experiment with it. If it really works much better than webkit, then one could evaluate it. But for the time being I think we should continue with webkit port.

@Ned
At this stage it might be better to move the discussion to the mailing list as oppose to turning this bug into a lengthy opinion based discussion thread.
Comment 11 Mikaël Barbero CLA 2016-09-26 06:58:10 EDT
Leo (as this bug is assigned to you) do you have an ETA for the port of SWT browser to webkit2gtk?
Comment 12 Leo Ufimtsev CLA 2016-09-26 10:27:44 EDT
(In reply to comment #11)
> Leo (as this bug is assigned to you) do you have an ETA for the port of SWT
> browser to webkit2gtk?

Thank you for inquiring. 

Stage 1 has been merged last week, (i.e, now 'execute()' works). Next I have to get 'evlaute()' working and fix up corner cases.

Due to the complexity, it's a bit hard to estimate how long this'll take. I would guess a month or two if things go well. 
This bug is currently at the center of my attention, i.e most of my time is spent on this bug. I know that in Fedora 25 Webkit1 get's deprecated, so I'm trying to get the port finished as soon as possible. 

~I'll keep you posted.
Comment 13 Christian Stadelmann CLA 2016-09-26 11:26:21 EDT
(In reply to Leo Ufimtsev from comment #12)
> (In reply to comment #11)
> > Leo (as this bug is assigned to you) do you have an ETA for the port of SWT
> > browser to webkit2gtk?
> 
> Stage 1 has been merged last week, (i.e, now 'execute()' works). Next I have
> to get 'evlaute()' working and fix up corner cases.

Basic functionality works in Fedora's upcoming 25 release (still in alpha) shipping a slightly modified eclipse 4.6.1¹. This is WebKit2 provided through the webkit2gtk4 package. Basic browser usage works fine:
* loading HTTP and HTTPS sites
* clicking URLs
* using inputs (search fields)
* playing HTML5 videos
* with both X11 and wayland backends
* scores 386/555 points in html5test.com

> Due to the complexity, it's a bit hard to estimate how long this'll take. I
> would guess a month or two if things go well. 
> This bug is currently at the center of my attention, i.e most of my time is
> spent on this bug. I know that in Fedora 25 Webkit1 get's deprecated, so I'm
> trying to get the port finished as soon as possible. 

No, WebKit1 has been deprecated for more than 3 years, by its maintainers, so basically everywhere. Fedora is planning to remove WebKit1 in Fedora 27², which I expect to be released in Q4/2017, so still 1 year from now.

¹ see https://pkgs.fedoraproject.org/cgit/rpms/eclipse.git/tree/?h=f25 for a list of changes ("patches") applied on fedora builds

² see https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/AKVB363GFCHHJ5MTHGVYHYT6NLLTF5VM/ and https://bugzilla.redhat.com/show_bug.cgi?id=1375784
Comment 14 Alexander Kurtakov CLA 2016-09-26 11:34:27 EDT
(In reply to Christian Stadelmann from comment #13)
> (In reply to Leo Ufimtsev from comment #12)
> > (In reply to comment #11)
> > > Leo (as this bug is assigned to you) do you have an ETA for the port of SWT
> > > browser to webkit2gtk?
> > 
> > Stage 1 has been merged last week, (i.e, now 'execute()' works). Next I have
> > to get 'evlaute()' working and fix up corner cases.
> 
> Basic functionality works in Fedora's upcoming 25 release (still in alpha)
> shipping a slightly modified eclipse 4.6.1¹. This is WebKit2 provided
> through the webkit2gtk4 package. Basic browser usage works fine:
> * loading HTTP and HTTPS sites
> * clicking URLs
> * using inputs (search fields)
> * playing HTML5 videos
> * with both X11 and wayland backends
> * scores 386/555 points in html5test.com

We are well aware of that and have changed the default in Fedora on purpose. Only some of the swt-js integration apis are not fully implemented and this is what Leo is working on. Although these are not used by IDEs in general the port will be considered done only when all APIs are fully functional.

> 
> > Due to the complexity, it's a bit hard to estimate how long this'll take. I
> > would guess a month or two if things go well. 
> > This bug is currently at the center of my attention, i.e most of my time is
> > spent on this bug. I know that in Fedora 25 Webkit1 get's deprecated, so I'm
> > trying to get the port finished as soon as possible. 
> 
> No, WebKit1 has been deprecated for more than 3 years, by its maintainers,
> so basically everywhere. Fedora is planning to remove WebKit1 in Fedora 27²,
> which I expect to be released in Q4/2017, so still 1 year from now.
> 
> ¹ see https://pkgs.fedoraproject.org/cgit/rpms/eclipse.git/tree/?h=f25 for a
> list of changes ("patches") applied on fedora builds
> 
> ² see
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/
> thread/AKVB363GFCHHJ5MTHGVYHYT6NLLTF5VM/ and
> https://bugzilla.redhat.com/show_bug.cgi?id=1375784
Comment 15 Leo Ufimtsev CLA 2016-09-26 12:39:22 EDT
(In reply to Christian Stadelmann from comment #13)

> Basic functionality works in Fedora's upcoming 25 release (still in alpha)
> shipping a slightly modified eclipse 4.6.1¹. This is WebKit2 provided
> through the webkit2gtk4 package. Basic browser usage works fine:
> * loading HTTP and HTTPS sites
> * clicking URLs
> * using inputs (search fields)
> * playing HTML5 videos
> * with both X11 and wayland backends
> * scores 386/555 points in html5test.com

Thank you for pointing those out. Figuring out what should work is part of the job.
I'm currently traversing the Browser jUnit test cases to see what else I should fix up. I'll tryna make sure the list there works also

> 
> ² see
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/
> thread/AKVB363GFCHHJ5MTHGVYHYT6NLLTF5VM/ and
> https://bugzilla.redhat.com/show_bug.cgi?id=1375784

Thank you for the reference, very helpful.
Comment 16 Mikaël Barbero CLA 2016-09-26 13:16:47 EDT
(In reply to Leo Ufimtsev from comment #12)
> ~I'll keep you posted.

Thank you for the update.
Comment 17 Leo Ufimtsev CLA 2017-02-16 12:43:35 EST
As side note, I experimented with authentication listener on Webkit2.
It seems to work:
	browser.addAuthenticationListener(event -> {
		System.out.println("Authentication request");
	});
	browser.setUrl("https://nominationsdemo.elia.be/B2C");
In that it get's triggered. But I don't know of a site and login credentials to verify that it can automatically log in. If someone knows a site, please feel free to let me know and I can verify that auto-login works on webkit2. But in theory it should.
Comment 18 Alexander Kurtakov CLA 2017-02-16 13:06:02 EST
(In reply to Leo Ufimtsev from comment #17)
> As side note, I experimented with authentication listener on Webkit2.
> It seems to work:
> 	browser.addAuthenticationListener(event -> {
> 		System.out.println("Authentication request");
> 	});
> 	browser.setUrl("https://nominationsdemo.elia.be/B2C");
> In that it get's triggered. But I don't know of a site and login credentials
> to verify that it can automatically log in. If someone knows a site, please
> feel free to let me know and I can verify that auto-login works on webkit2.
> But in theory it should.

I would assume HTTP authentication https://www.httpwatch.com/httpgallery/authentication/ Example10 will do it.
Comment 19 Leo Ufimtsev CLA 2017-02-16 14:38:41 EST
(In reply to Alexander Kurtakov from comment #18)
> I would assume HTTP authentication
> https://www.httpwatch.com/httpgallery/authentication/ Example10 will do it.

Thank you, this was useful. The authentication listener fills in user/password. Behaviour is the same for webkit1/2:
browser.addAuthenticationListener(event -> {
	System.out.println("Authentication request");
	event.user = "httpwatch";
	event.password = "foo";
});
browser.setUrl("https://www.httpwatch.com/httpgallery/authentication/#showExample10");
Comment 20 Leo Ufimtsev CLA 2017-05-01 13:10:00 EDT
Increasing importance. Webkit1 is being removed in upcoming Fedora versions.
Comment 21 Leo Ufimtsev CLA 2017-05-17 15:02:30 EDT
To be continued in 4.8 in:
Bug 516838 – [GTK3][webkit] Port SWT Browser to webkit2gtk 
https://bugs.eclipse.org/bugs/show_bug.cgi?id=516838
Comment 22 Guillermo Zunino CLA 2017-06-13 15:53:13 EDT
We have an answer! The great folks at Make Technology have built an SWT Chromium embed using the very latest version of Chromium. I've tested on win, mac, linux, and it works great on all 3 platforms.

The technical approach is very cool - use rust-bindgen to automatically wrap the Chromium Embedding Framework in rust bindings, then ffi_gen to automatically wrap the rust bindings with a Java interface.

Because so much of the code is autogenerated, the only code which will need maintenance from version to version are small bits of glue which connect these stubs.

For now, the browser just displays any URL - it doesn't support javascript interaction. But we are working on implementing the full SWT Browser API, and donating the entire codebase to eclipse under the EPL.

You can run the examples and browse the code at the repo here: https://github.com/maketechnology/cefswt

If you need this work, please consider giving a targeted donation to the Eclipse foundation. We've got a great start, but we will need more money to flesh out the full browser API and keep it maintained by the Eclipse foundation. Contact guillez@gmail.com and mikael@eclipse.org to arrange a donation.
Comment 23 Leo Ufimtsev CLA 2017-06-14 17:28:26 EDT
(In reply to Guillermo Zunino from comment #22)
> We have an answer! The great folks at Make Technology have built an SWT
> Chromium embed using the very latest version of Chromium. I've tested on
> win, mac, linux, and it works great on all 3 platforms.
> ...

That's quite interesting.