Re: [aether-users] SNAPSHOT artifacts: how to best access timestamped fi

[Andreas Sewe <andreas.sewe@xxxxxxxxxxxxxx>]

Hi Benjamin,

thank you so much for your detailed explanation. Much appreciated.

> There are various entities touched by a session with slightly different
> constraints:
> 
> In general, DefaultRepositorySystemSession itself is not thread-safe.
> But that shouldn't be an issue when properly used: The expectation is
> that a session gets set up once during some initialization phase and
> from there one, gets only read from. And mere reading by concurrent
> threads is no issue. So as long as configuring a session by thread A
> happens-before usage of the session by thread B, there's no danger to
> the session object itself.
> 
> The session holds a few mutable pieces like a cache and state. Those are
> backed by thread-safe implementations as required by API doc.
> 
> Any component referenced by the session which doesn't provide mutators
> is supposed to be thread-safe.
> 
> The only trouble with concurrency usually stems from the local
> repository and concurrent writes/reads to the filesystem. This is where
> the SyncContext comes generally into play but its current default impl
> is a noop. But similar to the session object itself, there's no harm
> with concurrency if only readers are involved.

One question about SyncContext: When an Artifact is acquired, I would
typically want to synchronize on it in some way. Now, this IMHO requires
some canonicalization, as to Artifact objects may refer to the same
artifact in the repo. For example, some thread has acquired a
timestamped Artifact already, when another wants to acquire the same
artifact with a "-SNAPSHOT" version suffix. Or on threads Artifact has a
file, the other one doesn't. Does some kind of Artifact canonicalization
logic exist already somewhere? Artifact.equals(..) is probably too picky
as to when artifacts are considered equal. Anyway, something like that
would IMHO be very useful, as it seems quite error-prone to do if you
are not an Aether insider (yet ;-).

>> - Given a "-SNAPSHOT" GAV, first check whether a timestamp artifact
>> exists in the local repo (presumably using resolution in offline mode).
>>
>> - If it does not exist locally, schedule resolution of the "-SNAPSHOT"
>> GAV in online mode in another thread.
>>
>> Now, I can guarantee that there is is only one online resolution job
>> running at any give time, but other threads might in the meantime
>> perform an offline-existance check using the local repo.
> 
> The online SNAPSHOT resolution eventually downloads newer metadata files
> to the local repo. With concurrent readers and writers, a reader could
> observe a partially written metadata file or observe an updated metadata
> file that points at a timestamped snapshots that is still being
> downloaded. So you need some form of synchronization to ensure proper
> operation here.

Yes, that's what I thought.

> As mentioned above, the current SyncContext impl is an empty
> placeholder. You could swap in your own SyncContextFactory that does the
> required locking for a given artifact (if all you do is invoke
> resolveArtifact(), per-artifact synchronization should be sufficient,
> metadata resolution is a nested call and is guarded by the
> artifact-level lock). Depending on the required granularity of
> concurrency, it might be easier for you to employ say a ReadWriteLock in
> your threads and synchronize at that coarse level.

OK, I think I understand SyncContext now.

Now, in my scenario, where snapshots are stored with timestamps on disc
and are thus never overwritten by Aether, I want all threads to still
obtain an "old" snapshot of an artifact will Aether resolves a newer
one. So, instead of blocking and waiting for the download thread to
finish, the other threads should simply pick the previous snapshot from
disc.

As far as I can tell, this is not what SyncContext is supposed to do.
So, I'll probably better keep track of all the "in-flight" artifact
resolutions myself and answer with a cached, old Artifact until the
download thread finishes and updates the cache.

Or does Aether have an internal cache that could kick in such a case,
i.e., while the on-disc metadata is currently being written? If not, no
problem. I think my "in-flight" cache would work quite well. Haven't had
time to implement it in Code Recommenders, though. :-(

> Hope that lengthy answer gets you further.

Yes, it does. Thank you very much.

Andreas
-- 
Codetrails UG (haftungsbeschränkt)
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Mobile: +49-170-811-3791
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940