[platform-core-dev] Comments to "Marker Position Updates"

[Kai-Uwe_Maetzel@xxxxxxx]

My comments are based on my experience gained with text editors. Text
editors change the content of a file and update all text markers of this
file accordingly. The model is basically the following:

- The editor fetches the content of a text file and puts it into a buffer.
- The editor retrieves all markers attached to the text file.
- For all text markers the editor creates  an in-memory representation (aka
positions) reflecting the type of the represented marker.
- The editor is configured with a set of position updaters each responsible
for one or more marker types. Per marker type there can be a list of
updaters. The updaters are called in the sequence they appear in the list.
- Each text manipulation in the editor is a replace command (offset,
length, replace string).
- On each text manipulation all position updaters are notified.  The
notification carries the replace command. The position updaters adapt the
positions' offsets and lengths, the underlying markers remain untouched.
- The editor is configured with a set of marker updaters each responsible
for one or more marker types
- On save the editor updates the file's content and calls all registered
marker updaters passing in the marker to be updated as well as the position
representing the marker (i.e. the "would be" location of the marker
resulting from the text manipulations). The updaters adapt the marker
attributes. When all updaters are done, the changed markers are committed
to the file. All this is done in a single operation, i.e. content and
markers are always updated together in one "transaction".

What's the intent of supporting different types of position updaters? If a
position is, say, at offset 3, inserting text at 3 can, e.g., either shift
or stretch the position. What happens is application specific. Thus,
applications must have a way to define how to update their positions.

What's the intent of supporting different types of marker updaters? If a
text marker's attributes for start, end, and line changes all attributes
whose values are derived from the position and the content must also be
updated. As the set of marker types is extensible, the set of marker
updaters must also be extensible.

Why is there a concept of marker updaters at all? Marker updaters exist to
ensure the transactional nature of content and marker changes. Even if you
don't buy the need for a transactional change, there are two major issues
which they are solving: a) Updating markers in a resource change listener
forces you to find a way to apply your changes to the marker in an
operation that is executed after the running operation notifying you as the
workspace is currently locked. If you register as a listener that gets
write access to the workspace, you might not have access to the information
you need as others are not yet updated. I don't know of any good support to
"post an operation behind the currently running operation". b) Having
solved a) for each marker update all resource changed listeners are called.
This is a performance issue.


Having described how the text infrastructure deals with markers, here my
comments to the proposal.

My intention of having a core concept is to get rid of the basic marker
updater infrastructure in the text infrastructure. The text infrastructure
is to high up for general marker updating but it is the right place of
updating text markers.

- The core should not be aware of text markers, as Jeff already mentioned.

- Content and marker updating should be transactional. The proposal pushes
the burden of ensuring this to the client. Thus, in this respect the
proposal does not offer an advantage. A typical client code would create a
workspace runnable and call IFile.setContent and IFile.updateMarkers
directly after each other.

- The current proposal does not work for clients that need to
transactionally apply multiple changes to a file. This includes editors. As
described above, editors perform a potentially big number of replace
operations before saving. Calling IFile.updateMarker after each replace
operation would update the markers although the content of the file is
still the old. If another client retrieves all markers and evaluates them,
it will stumble over markers with invalid positions because they refer to a
content which is still in a buffer and not yet on the file system. A
potential solution would be to change IFile.updateMarker in a way that it
would accept a edit script, i.e. a list of replace commands rather than a
single replace command.  This is a potentially dangerous solution. It
introduces the problem of unbounded memory consumption. Just think of a big
number of replaces of the whole content of the editor which large pieces of
text, say 500 x 1MB.


To me it seems that a text independent solution in core would fit better
with the architecture we already have. Here a raw picture of how it might
look like:
As the original proposal describes, marker updaters can be registered per
marker type. Potentially more than one per marker types are allowed as long
as the attribute sets manipulated by these updaters are disjoint. IFile
does not provide an updateMarker method. Instead on setContent and
refreshFromLocal all marker updaters are called. A called updater gets
access to the new and the old content of the resource. Additionally, it
gets the marker as parameter. There must be a way for updaters to indicate
that a marker gets removed or stale in result of the updating.

This way, each level can install updaters for the marker types it
introduces or knows of. The text infrastructure could provide text marker
updaters leading to the nice-to-have request of dynamic registration and
unregistration of marker updaters. An editor, e.g., could temporarily
register itself as marker updater as it already knows the new position of
the markers. In addition, a more general text marker updater, e.g., could
diff the new and the old content and use the diff for adapting markers.
Right now, this does not solve the headless case yet - but it shows the
need for a headless text infrastructure.

Kai