Hi John,
I've followed these steps and I still find issues when trying to update metastore.json from several processes.
In my opinion the problem is in SimpleMetastore.java.
For example:
Process 1 wants to delete proj1
Process 2 wants to delete proj2
1.
Process 1 calls SimpleMetastore.deleteProject(proj1)
a.
User lock is set
b.
metastore.json is read (fileLocker is set and released)
c.
proj1 is deleted from file system
d.
proj1 is deleted from local copy of metastore.json
e.
metastore.json is updated in file system ((fileLocker is set and released))
Process 1 overrides changes from Process 2
f.
User lock is released
2.
Process 2 calls SimpleMetastore.deleteProject(proj2)
a.
User lock is set (no problem because is another process)
b.
metastore.json is read (fileLocker is set and released)
c.
proj2 is deleted from file system
d.
proj2 is deleted from local copy of metastore.json
e.
metastore.json is updated in file system ((fileLocker is set and released))
before Process 1 updates metastore
f.
User lock is released
I don't think the above mentioned scenario is supported by Orion locking mechanism. In order to support this scenario User lock should be cross process (e.g.: FileLocker on User.json).
Best Regards
Sergio
From:
orion-dev-bounces@xxxxxxxxxxx [mailto:orion-dev-bounces@xxxxxxxxxxx]
On Behalf Of John Arthorne
Sent: יום ד 21 אוקטובר 2015 20:28
To:
orion-dev@xxxxxxxxxxx
Subject: Re: [orion-dev] Orion high availability support
This is certainly supported, but how to do it is not very well documented. I found this blog post that gives some of the explanation:
I have just copied some of the core information into the orion server admin guide:
If you follow these steps and still see problems, please follow up here with more questions!
----- Original message -----
From: "Rozenszajn, Sergio" <sergio.rozenszajn@xxxxxxx>
Sent by: orion-dev-bounces@xxxxxxxxxxx
To: "orion-dev@xxxxxxxxxxx" <orion-dev@xxxxxxxxxxx>
Cc:
Subject: [orion-dev] Orion high availability support
Date: Wed, Oct 21, 2015 2:40 AM
Hi,
We are trying to implement Orion 8 with high availability. We will have multiple VM's that run Orion with a shared file system where the workspace files are maintained. We have turned on FileLocker.locking.
In one of our automatic tests we log on with same user id from different sessions/different VM's. In this test each session tries to create multiple projects in parallel.
As a result we get this error:
2015 10 21 05:40:09#+00#ERROR#org.eclipse.orion.server.config##anonymous#http-bio-8041-exec-6#na#hcproxy#orion#web#hcproxy#Meta File Error, cannot read JSON file /mnt/perm_storage/persistent/orion_web/xe/xeee1cc67/P9/P900902/xeee1cc67$P900902-OrionContent.json
from disk, reason: A JSONObject text must begin with '{' at character 0
When doing the same from a single VM, it works fine.
I assume that we have here a locking issue when updating metastore.json file from different processes.
I was planning to use a FileLocker lock in SimpleMetaStore.java, instead of the ReadWriteLock to have a lock on file level that protects from cross process access.
The question is if you have already noticed this issue and if there is a proposed solution.
Best Regards
Sergio Rozenszajn
