Bug 436378 - [repository] Cannot load the same repo in 2 parallel threads/jobs
Summary: [repository] Cannot load the same repo in 2 parallel threads/jobs
Status: RESOLVED WORKSFORME
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.10.0 Luna   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2014-06-02 11:17 EDT by Mickael Istria CLA
Modified: 2020-02-22 07:41 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mickael Istria CLA 2014-06-02 11:17:49 EDT
Calling MetadataManager.loadRepository(aRepoUri) from 2 jobs/threads with the same "aRepoUri" ends with conflict on filesystem that messes up the cache of the repository and makes most operation on this repository fail after that.

Here is what this failure looks like in the log
  org.eclipse.equinox.p2.core.ProvisionException: An error occurred while downloading https://devstudio.jboss.com/updates/8.0.0/jboss-devstudio-8.0.0.Beta2-updatesite-core/content.jar. The cache file /home/mistria/jbdevstudio-8.0.0.Beta2a (copy).bk/studio/p2/org.eclipse.equinox.p2.repository/cache/downloading/content207258997.jar could not be renamed to /home/mistria/jbdevstudio-8.0.0.Beta2a (copy).bk/studio/p2/org.eclipse.equinox.p2.repository/cache/downloading/content207258997.jar.
        at org.eclipse.equinox.internal.p2.repository.CacheManager.updateCache(CacheManager.java:371)
        at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:208)
        at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
        at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
Comment 1 Ed Merks CLA 2020-02-22 07:41:19 EST
I cannot reproduce this.  I have a "test" like this:

  private void test() throws Exception
  {
    final java.net.URI uri = new java.net.URI("http://download.eclipse.org/releases/2020-03");
    final AtomicBoolean wait = new AtomicBoolean(true);
    final AtomicInteger count = new AtomicInteger();
    for (int i = 0; i < 10; ++i)
    {
      new Thread()
      {
        @Override
        public void run()
        {
          while (wait.get())
          {
          }
          try
          {
            System.err.println("Starting");
            manager.loadRepository(uri, new NullProgressMonitor());
            System.err.println("Finished");
          }
          catch (Exception ex)
          {
            ex.printStackTrace();
            throw new RuntimeException(ex);
          }
          finally
          {
            count.incrementAndGet();
          }
        }
      }.start();
    }
    wait.set(false);

    while (count.get() < 10)
    {
      Thread.sleep(1000);
    }
  }

What happens is that each each thread goes into the method org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(URI, IProgressMonitor, String, int)

	protected IRepository<T> loadRepository(URI location, IProgressMonitor monitor, String type, int flags) throws ProvisionException {
		checkValidLocation(location);
		SubMonitor sub = SubMonitor.convert(monitor, 100);
		boolean added = false;
		IRepository<T> result = null;

		try {
			enterLoad(location, sub.newChild(5));
			result = basicGetRepository(location);
			if (result != null)
				return result;
			if (checkNotFound(location))
				fail(location, ProvisionException.REPOSITORY_NOT_FOUND);
			//add the repository first so that it will be enabled, but don't send add event until after the load
			added = addRepository(location, true, false);

But only one thread gets past the enterLoad call initially.  That thread does all the loading.  When that thread calls exitLoad, another thread leaves enterLoad and calls basicGetRepository, returning a non-null result causing it to return immediately.

In no case do multiple threads end up loading the same URI at the same time.

If there is a problem, I would need to understand how to reproduce it.