Re: [platform-swt-dev] Chromium Browser

[Aleksandar Kurtakov <akurtako@xxxxxxxxxx>]

On Mon, Nov 13, 2017 at 12:49 AM, Guillermo Zunino <guillez@xxxxxxxxx> wrote:
> Hi SWT Team!

Hi Guillermo,
My main concern is maintenance, so whenever I ask a question don't
take it as advertisement of one approach or bashing another - it's
simply trying to keep consistency in the project so if/when this
enters SWT it's smth maintainable in the long term and not one time
thing which can't be easily changed and adopted to new CEF or
toolchain.... And browsers are known to be especially bad in stability
so my concerns are just bigger for anything related to them.
See inlined comments for details.

>
> I'm Guillermo Zunino (from Make Technology, http://wemaketechnology.com) and
> project https://github.com/maketechnology/cefswt.
> Mikael Barbero talked about the project with some of you during EclipseCon
> Europe and asked me to write to the list. As you show willingness to
> integrate the work in SWT, I'm writing to start discussing some of the
> challenges and to get your feedback, opinions and ideas.
> Here is a post on the subject by Mikael
> https://medium.com/@mikael.barbero/chromium-eclipse-swt-integration-c61f416e97d1
> and here is more information of the background, design and original plan
> (although the idea now is to integrate it to SWT instead of nebula if
> possible):
> https://docs.google.com/document/d/1xpLxquAZd6SGDK7XvYHoj2g_PBnnBX8SFhgwIJPv7lg/edit?usp=sharing
>
> I describe below an overview and some of the points which I believe are the
> biggest challenges.
> Please ask if you want to know more about the selected tools and libraries,
> or any other technical or non technical question.
>
> - Design:
> Currently the widget tries to match SWT browser API but (for now) is a
> different widget component in its own bundle. The widget talks to a small
> integration layer (native lib) using JNR-FFI (instead of JNA/JNI). And this
> native layer talks to CEF framework C APIs.

I fully understand the preference to use JNR over plain JNI. But SWT
has a design requirement that it should be usable as standalone
graphics library for apps without enforcing any other dependencies.
While this can be discussed it also opens the question of using
multiple means to achieve similar results in one project. Additionally
SWT has a generator for JNI which hides a lot of the JNI complexity so
it's by no means like working with plain JNI. To the state people
extracted in standalone one see http://fusesource.github.io/hawtjni/
for details. Have you tried this approach? If yes, what were the
showstoppers. I don't imply it's better or even comparable to JNR but
for the means of the bindings as used in SWT it's quite sufficient.


> There are 3 bundle fragments which provide the binaries for each platform
> (mac64, win64, linux64).
> The native lib is written in Rust language the same as a helper executable
> required by CEF to spawn subprocess (CEF is multiprocess).

This is yet another concern for me. Rust has it's advantages but this
means current maintainers to learn one more language, deal with yet
another buildsystem(cargo), maintain yet another toolchain on all the
different platforms for the native builders and etc. And this
toolchain is evolving quite fast now IIRC. So keeping up with that
will be constant drain of resources. Do you plan on maintaining it in
the long term?
While trying compiling https://github.com/maketechnology/cefrust I
noticed a real showstopper from Linux POV. It compiles against GTK2
while SWT by default uses GTK3 and there is noone supporting GTK2
port. Mixing GTK 2 and 3 in the same app eventually leads to crashes.
Chrome(ium) has GTK3 port since 59.x why is it not used? I would guess
there are reasons for that but GTK2 is on it's path out of SWT so
adding a module with hard dependency on it seems a no-go.

>
> - Binaries (and sizes):
>
> cefrustlib (.so or .dll)  ~2MB
> Native layer binding to CEF APIs
> We should be able to shrink this to less than 1MB after optimizations.
> cefrust_subp (.exe) ~0.5MB
> Helper to spawn CEF subprocesses.
> libcef (.so, .dll or .framework) ~120MB
> CEF shared lib. No debugging symbols.
> cef resources ~25MB
> Non localized resources used by CEF. Some may not be entirely required
> (depending on browser usage), or could be delivered as separate fragments,
> for example devtools (5MB) and cef_extensions (3,5MB)
> cef locales (15MB)
> Localized resources used by CEF
>
> Our native lib and subprocess helper are included in the git repository
> (using git lfs). The rust source code for those is in separate git repo
> (https://github.com/maketechnology/cefrust/).
> The cef binaries are not part of the git repo but downloaded as part of the
> build from http://opensource.spotify.com/cefbuilds/index.html.
> The resulting built fragments, as compressed .jars with all binaries
> mentioned above inside, are about ~62MB for each platform.

These sizes are really huge so they can't be shipped as part of swt.
There must be some way to ship it separately as an optional part of
SWT. Having part of it in SWT with some clever mechanics to redirect
work to a fragment seems the likely way to do it.

>
> - Build time dependencies
>
> Rust (used to code and build the native lib and helper)
>
> This is actually not required during the bundle and fragments build, as the
> shared libraries are part of the git repo.

Even if part of the repo there is a certain group of contributors that
build Eclipse SDK from source incl. the native parts and the ease of
setting up for this group is always important.

>
> Gradle (to automate the many build steps)
Certain part of your gradle script would have to be rewritten in maven
way to integrate with existing swt build system.

> Tycho (to build the bundle, fragments and p2 repo)
> Jboss Reddeer (or SWT Bot) for testing
> JUnit
>
> - Run time dependencies and licenses:
>
> SWT
> JNR-FFI (Apache 2.0) (already in Orbit, but an older version, we are
> depending on latest)
That would require CQ and Orbit build at some point if this is a must
have, although I would prefer to not ship jnr with swt.

>
> ASM (BSD) (already in Orbit, existing version should work)
By exisging version do you mean 6.0 (as this is the one the eclipse
platform dev stream uses)?

>
> CEF (BSD) (bundled in our fragments)
> Native libs (coded in Rust, we use some third party libs from crates.io to
> develop it)
>
> rust std lib (MIT/Apache-2.0)
> libc (MIT/Apache-2.0)
> X11 (CC0-1.0)
> nix (MIT)
> winapi/user32 (MIT)
> cocoa (MIT / Apache-2.0)

these shouldn't be problematic from legal POV but they will require
CQs too. These are all linked in the same library, right?

>
> - CEF lifecycle
>
> The first created Chromium browser extracts the binaries to ~/.swtcef
> (similar to how swt does) and initializes CEF (renderer and gpu subprocess
> are started). CEF has a memory footprint of ~20MB per process plus the web
> pages.
> A new subprocess is started for each different domain url.
> CEF is shutdown using a java shutdown hook or explicitly by the client.
> Can't be shutdown after the last browser is closed, due it doesn't support
> reinitialization. This one is tricky, because shutdown is not a SWT Browser
> API, and CEF shutdown must be called in the main thread. The java hook is
> called too late sometimes. For eclipse rcp, is better to explicitly call
> shutdown on app exit hook.
>
> - CEF versioning
>
> Currently we are targeting a single CEF/Chromium version, 59 and should be
> able to update to 60, 61 and 62 without major issues based on our analysis.
> And we try to consume pre-built binaries form
> http://opensource.spotify.com/cefbuilds/index.html. We also have a single p2
> repo.
> But I see somewhere in the future that we may be forced to support multiple
> CEF versions and build CEF ourself. For example if CEF drops GTK2 support,
> we may have to provide separate CEF binaries (maybe targeting different CEF
> versions also depending on API breakage) compatible with gtk2 and gtk3.
> Similar could happen when Wayland support lands on Chromium and CEF.
> Things are not so clear for me here as it depends on upstream project
> decisions (Chromium and CEF).

IMHO, providing GTK3 only is the way to go as GTK2 will soon be dead
from SWT POV. So even is Chromium has GTK3 support CEF still not has
it? Am I reading this well? With GTK3 providing switchable backends
(x11 or wayland) SWT has no issues with Wayland or X11 other than the
places where the swt code is using x11 api directly.

>
> - Integration
>
> I think the best developer experience, would be to provide this as a new
> SWT.CEF (or SWT.CHROMIUM) constant and new subclass of
> org.eclipse.swt.browser.WebBrowser. The full browser API is planned to be
> implemented and tested using the existing SWT browser test suite.
> The small integration libs can be put into eclipse.platform.swt.binaries.git
> repo maybe.

How small are they?
In order for integration libs to enter swt.binaries we must know that
producing them is properly hooked in the existing build system
including  needed toolchains available on build servers for
consistency of the builds.

> Given the size of CEF binaries probably they should be continue to be
> download during build (http://opensource.spotify.com/cefbuilds/index.html is
> the "official" release channel of CEF and they keep all historical
> versions).

That would be a legal issue for SWT as we will need IP team to approve
every version we use.

>
> - Distribution
>
> Distribution wise I envision this as separate .jars from main swt .jar given
> the size of the binaries and the third party dependencies. The java
> implementation could be part of main swt, and the native binaries as
> separate .jars per platform (valid osgi fragments). This should allow for
> plain java apps, to include the desired .jar and for osgi apps to include
> the specific fragment. The java implementation can check for the existence
> of natives an fail fast if couldn't find it.
> This way there is no new dependency introduced to SWT and users wishing to
> use Chromium should opt-in for it, instead of making everyone pay the size
> and third party dependencies cost.

I like the approach of separate download, the usual approach is to
fallback to provide maximum feature to users when possible. E.g. when
one requests SWT.CHROMIUM Browser and the fragment is not availabla a
warning should be logged in some way but the app shoud not fail but
rather the OS browser will be instantiated and returned.

>
> As said the idea here is to trigger discussion and know your opinions to
> move forward.

Have you tackled the hard features like Java<->JS interaction? As we
are in the process of migrating to Webkit2 API it's these features
that takes times more time than the initial port to just embed the
html renderer. I'm interested in seeing a list of missing features(API
not implemented) to get a feeling of how featurefull the port is.

>
> Best regards, and sorry for the large email!
>
> --
>
> Guillermo Zunino
> guillez@xxxxxxxxx
>
>
> _______________________________________________
> platform-swt-dev mailing list
> platform-swt-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/platform-swt-dev



-- 
Alexander Kurtakov
Red Hat Eclipse Team