Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [equinox-dev] Re: Patch for RepoOptimizer


Ok.  So for now I am thinking that the optimizers are just independent.  When/if we see a common pattern we can look at generalizing and putting some infrastructure.  So far the only win I've seen is the potential for caching if you are doing multiple optimzations.  

Having said that, if we go for different bundles for each optmizer, the current one should be renamed to be a pack200 optimizer.  Should I do that?

Jeff



Stefan Liebig <Stefan.Liebig@xxxxxxxxxxxx>
Sent by: equinox-dev-bounces@xxxxxxxxxxx

10/24/2007 02:43 AM

Please respond to
Equinox development mailing list <equinox-dev@xxxxxxxxxxx>

To
Equinox development mailing list <equinox-dev@xxxxxxxxxxx>
cc
Subject
Re: [equinox-dev] Re: Patch for RepoOptimizer





Yes, that´s right. There is not that much common code. It is mainly parsing the command line arguments. Dealing with different types of inputs (special parameters for the delta thing) is surely easier if it is not generalized but done by each optimizer individually.
Should each optimizer (or strongly related optimizers) have it´s on bundle?


Jeff McAffer wrote:


<moving this thread to equinox-dev>


I guess I wonder if there is much value in generalizing the optimizer structure.  For example, what is the difference between running the pack200 optimizer then running the delta optimizer and running one app and spec'ing multiple optimizers?  I am concerned actually that the different optimzers will require significantly different inputs (e.g., the delta optimizer is going to need some baselines etc).  As yet there does not appear to be much common code to be shared.


So for now I was thinking of just having separate optimizers...


One place that we might be able to win is if we could somehow coordinate the dowload of the "canonical" artifact form (e.g., the unadulterated JAR).  It would be boring to do several optimizations remotely and have the artifact downloaded multiple times.

Sorry, I did not get the general picture here. Could you please rephrase it?

Stefan


What do you think?


Jeff



Stefan Liebig <Stefan.Liebig@xxxxxxxxxxxx>

10/18/2007 09:43 AM


To
Jeff McAffer/Ottawa/IBM@IBMCA
cc
Subject
Patch for RepoOptimizer







Hi Jeff,

I tried to generalize the handling of  ´optimizers´. The idea is that
this forms the basis for extending the Optimizer with new IOptimizer
impls. Maybe we have to change a few names. Test works as before.
Tell me your opinion.


Tschüß,
Stefan

### Eclipse Workspace Patch 1.0
#P org.eclipse.equinox.p2.repositoryoptimizer
Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java
===================================================================
RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.repositoryoptimizer/src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java,v
retrieving revision 1.1
diff -u -r1.1 Pack200Step.java
--- src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java                 17 Oct 2007 03:47:55 -0000                 1.1
+++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java                 18 Oct 2007 13:34:47 -0000
@@ -90,7 +90,7 @@
                                 // So before closing, run unpack and write the unpacked result to the destination
                                 performPack();
                                 super.close();
-                                  // TODO need to get real status here but curently the JAR processor does not give
+                                  // TODO need to get real status here but currently the JAR processor does not give
                                 // any reasonable return status
                                 if (status == null)
                                                  status = Status.OK_STATUS;
Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java
===================================================================
RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.repositoryoptimizer/src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java,v
retrieving revision 1.1
diff -u -r1.1 Optimizer.java
--- src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java                 17 Oct 2007 03:47:55 -0000                 1.1
+++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java                 18 Oct 2007 13:34:47 -0000
@@ -8,69 +8,33 @@
******************************************************************************/
package org.eclipse.equinox.internal.p2.repositoryoptimizer;

-import java.io.IOException;
-import java.io.OutputStream;
-import org.eclipse.core.runtime.NullProgressMonitor;
-import org.eclipse.equinox.p2.artifact.repository.*;
-import org.eclipse.equinox.p2.artifact.repository.processing.*;
+import java.util.List;
+import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository;
import org.eclipse.equinox.p2.metadata.IArtifactKey;

public class Optimizer {
                private IArtifactRepository repository;
+                 private List optimizers;

-                 public Optimizer(IArtifactRepository repository) {
+                 public Optimizer(IArtifactRepository repository, List optimizers) {
                                 this.repository = repository;
+                                  this.optimizers = optimizers;
+
+                                  for (int i = 0; i < optimizers.size(); i++) {
+                                                   IOptimizer optimizer = (IOptimizer) optimizers.get(i);
+                                                   optimizer.setArtifactRepository(repository);
+                                  }
                }

                public void run() {
                                 IArtifactKey[] keys = repository.getArtifactKeys();
                                 for (int i = 0; i < keys.length; i++) {
                                                  IArtifactKey key = keys[i];
-                                                   if (!key.getClassifier().equals("plugin"))
-                                                                    continue;
-                                                   IArtifactDescriptor[] descriptors = repository.getArtifactDescriptors(key);
-                                                   IArtifactDescriptor complete = null;
-                                                   boolean optimized = false;
-                                                   for (int j = 0; j < descriptors.length; j++) {
-                                                                    IArtifactDescriptor descriptor = descriptors[j];
-                                                                    if (descriptor.getProcessingSteps().length == 0)
-                                                                                     complete = descriptor;
-                                                                    optimized |= isOptimized(descriptor);
+                                                   for (int j = 0; j < optimizers.size(); j++) {
+                                                                    IOptimizer optimizer = (IOptimizer) optimizers.get(j);
+                                                                    optimizer.optimize(key);
                                                  }
-                                                   if (!optimized)
-                                                                    optimize(complete);
                                 }
                }

-                 private void optimize(IArtifactDescriptor descriptor) {
-                                  ArtifactDescriptor newDescriptor = new ArtifactDescriptor(descriptor);
-                                  ProcessingStepDescriptor[] steps = new ProcessingStepDescriptor[] {new ProcessingStepDescriptor("org.eclipse.equinox.p2.repositoryoptimizer.Pack200Unpacker", null, true)};
-                                  newDescriptor.setProcessingSteps(steps);
-                                  OutputStream repositoryStream = null;
-                                  try {
-                                                   repositoryStream = repository.getOutputStream(newDescriptor);
-
-                                                   // Add in all the processing steps needed to optimize (e.g., pack200, ...)
-                                                   ProcessingStepHandler handler = new ProcessingStepHandler();
-                                                   OutputStream destination = handler.link(new ProcessingStep[] {new Pack200Step()}, repositoryStream, null);
-
-                                                   // Do the actual work by asking the repo to get the artifact and put it in the destination.
-                                                   repository.getArtifact(descriptor, destination, new NullProgressMonitor());
-                                  } finally {
-                                                   if (repositoryStream != null)
-                                                                    try {
-                                                                                     repositoryStream.close();
-                                                                    } catch (IOException e) {
-                                                                                     // TODO Auto-generated catch block
-                                                                                     e.printStackTrace();
-                                                                    }
-                                  }
-                 }
-
-                 private boolean isOptimized(IArtifactDescriptor descriptor) {
-                                  // TODO this is a hack for now.  Really we should inspect the steps to see
-                                  // if the optimization(s) we are doing here have already been done
-                                  return descriptor.getProcessingSteps().length > 0;
-                 }
-
}
Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java
===================================================================
RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.repositoryoptimizer/src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java,v
retrieving revision 1.1
diff -u -r1.1 Application.java
--- src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java                 17 Oct 2007 03:47:55 -0000                 1.1
+++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java                 18 Oct 2007 13:34:47 -0000
@@ -9,7 +9,7 @@
package org.eclipse.equinox.internal.p2.repositoryoptimizer;

import java.net.URL;
-import java.util.Map;
+import java.util.*;
import org.eclipse.equinox.app.IApplication;
import org.eclipse.equinox.app.IApplicationContext;
import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository;
@@ -22,12 +22,13 @@
public class Application implements IApplication {

                private URL artifactRepositoryLocation;
+                 private List optimizers = new ArrayList();

                public Object start(IApplicationContext context) throws Exception {
                                 Map args = context.getArguments();
                                 initializeFromArguments((String[]) args.get("application.args"));
                                 IArtifactRepository repository = setupRepository(artifactRepositoryLocation);
-                                  new Optimizer(repository).run();
+                                  new Optimizer(repository, optimizers).run();
                                 return null;
                }

@@ -47,8 +48,8 @@
                                                  return;
                                 for (int i = 0; i < args.length; i++) {
                                                  // check for args without parameters (i.e., a flag arg)
-                                                   //                                                   if (args[i].equals("-pack"))
-                                                   //                                                                    pack = true;
+                                                   if (args[i].equalsIgnoreCase("-pack200"))
+                                                                    optimizers.add(new Pack200Optimizer());

                                                  // check for args with parameters. If we are at the last argument or
                                                  // if the next one has a '-' as the first character, then we can't have
@@ -57,7 +58,7 @@
                                                                   continue;
                                                  String arg = args[++i];

-                                                   if (args[i - 1].equalsIgnoreCase("-artifactRepository") | args[i - 1].equalsIgnoreCase("-ar"))
+                                                   if (args[i - 1].equalsIgnoreCase("-artifactRepository") || args[i - 1].equalsIgnoreCase("-ar"))
                                                                   artifactRepositoryLocation = new URL(arg);
                                 }
                }
Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java
===================================================================
RCS file: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java
diff -N src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java
--- /dev/null                 1 Jan 1970 00:00:00 -0000
+++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java                 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,22 @@
+/*******************************************************************************
+ * Copyright (c) 2007 compeople AG and others.
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at
+ *
http://www.eclipse.org/legal/epl-v10.html
+ *
+ * Contributors:
+ *                  compeople AG (Stefan Liebig) - initial API and implementation
+ *******************************************************************************/
+package org.eclipse.equinox.internal.p2.repositoryoptimizer;
+
+import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository;
+import org.eclipse.equinox.p2.metadata.IArtifactKey;
+
+public interface IOptimizer {
+
+                 void setArtifactRepository(IArtifactRepository repository);
+
+                 void optimize(IArtifactKey key);
+
+}
Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java
===================================================================
RCS file: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java
diff -N src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java
--- /dev/null                 1 Jan 1970 00:00:00 -0000
+++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java                 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,32 @@
+/*******************************************************************************
+ * Copyright (c) 2007 compeople AG and others.
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at
+ *
http://www.eclipse.org/legal/epl-v10.html
+ *
+ * Contributors:
+ *                  compeople AG (Stefan Liebig) - initial API and implementation
+ *******************************************************************************/
+package org.eclipse.equinox.internal.p2.repositoryoptimizer;
+
+import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository;
+
+/**
+ * Forms a common root for various <code>IOptimizers</code>.
+ * An <code>IOptimizers</code> does not need to implement this but doing so
+ * eases writing of <code>IOptimizers</code>.
+ */
+public abstract class AbstractOptimizer {
+
+                 protected IArtifactRepository repository;
+
+                 public AbstractOptimizer() {
+                                  super();
+                 }
+
+                 public void setArtifactRepository(IArtifactRepository repository) {
+                                  this.repository = repository;
+                 }
+
+}
Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java
===================================================================
RCS file: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java
diff -N src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java
--- /dev/null                 1 Jan 1970 00:00:00 -0000
+++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java                 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,72 @@
+/*******************************************************************************
+* Copyright (c) 2007 compeople AG and others.
+* All rights reserved. This program and the accompanying materials
+* are made available under the terms of the Eclipse Public License v1.0
+* which accompanies this distribution, and is available at
+*
http://www.eclipse.org/legal/epl-v10.html
+*
+* Contributors:
+*                  compeople AG (Stefan Liebig) - initial API and implementation
+*  IBM - continuing development
+*******************************************************************************/
+package org.eclipse.equinox.internal.p2.repositoryoptimizer;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import org.eclipse.core.runtime.NullProgressMonitor;
+import org.eclipse.equinox.p2.artifact.repository.ArtifactDescriptor;
+import org.eclipse.equinox.p2.artifact.repository.IArtifactDescriptor;
+import org.eclipse.equinox.p2.artifact.repository.processing.*;
+import org.eclipse.equinox.p2.metadata.IArtifactKey;
+
+public class Pack200Optimizer extends AbstractOptimizer implements IOptimizer {
+
+                 public void optimize(IArtifactKey key) {
+                                  if (!key.getClassifier().equals("plugin"))
+                                                   return;
+
+                                  IArtifactDescriptor[] descriptors = repository.getArtifactDescriptors(key);
+                                  IArtifactDescriptor complete = null;
+                                  boolean optimized = false;
+                                  for (int j = 0; j < descriptors.length; j++) {
+                                                   IArtifactDescriptor descriptor = descriptors[j];
+                                                   if (descriptor.getProcessingSteps().length == 0)
+                                                                    complete = descriptor;
+                                                   optimized |= isOptimized(descriptor);
+                                  }
+                                  if (!optimized)
+                                                   optimize(complete);
+                 }
+
+                 private void optimize(IArtifactDescriptor descriptor) {
+                                  ArtifactDescriptor newDescriptor = new ArtifactDescriptor(descriptor);
+                                  ProcessingStepDescriptor[] steps = new ProcessingStepDescriptor[] {new ProcessingStepDescriptor("org.eclipse.equinox.p2.repositoryoptimizer.Pack200Unpacker", null, true)};
+                                  newDescriptor.setProcessingSteps(steps);
+                                  OutputStream repositoryStream = null;
+                                  try {
+                                                   repositoryStream = repository.getOutputStream(newDescriptor);
+
+                                                   // Add in all the processing steps needed to optimize (e.g., pack200, ...)
+                                                   ProcessingStepHandler handler = new ProcessingStepHandler();
+                                                   OutputStream destination = handler.link(new ProcessingStep[] {new Pack200Step()}, repositoryStream, null);
+
+                                                   // Do the actual work by asking the repo to get the artifact and put it in the destination.
+                                                   repository.getArtifact(descriptor, destination, new NullProgressMonitor());
+                                  } finally {
+                                                   if (repositoryStream != null)
+                                                                    try {
+                                                                                     repositoryStream.close();
+                                                                    } catch (IOException e) {
+                                                                                     // TODO Auto-generated catch block
+                                                                                     e.printStackTrace();
+                                                                    }
+                                  }
+                 }
+
+                 private boolean isOptimized(IArtifactDescriptor descriptor) {
+                                  // TODO this is a hack for now.  Really we should inspect the steps to see
+                                  // if the optimization(s) we are doing here have already been done
+                                  return descriptor.getProcessingSteps().length > 0;
+                 }
+
+}
### Eclipse Workspace Patch 1.0
#P org.eclipse.equinox.p2.tests
Index: src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java
===================================================================
RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.tests/src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java,v
retrieving revision 1.1
diff -u -r1.1 PackRepositoryTest.java
--- src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java                 17 Oct 2007 21:12:34 -0000                 1.1
+++ src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java                 18 Oct 2007 13:38:41 -0000
@@ -2,10 +2,13 @@

import java.io.*;
import java.net.URL;
+import java.util.ArrayList;
+import java.util.List;
import junit.framework.TestCase;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.equinox.internal.p2.metadata.ArtifactKey;
import org.eclipse.equinox.internal.p2.repositoryoptimizer.Optimizer;
+import org.eclipse.equinox.internal.p2.repositoryoptimizer.Pack200Optimizer;
import org.eclipse.equinox.p2.artifact.repository.*;
import org.eclipse.equinox.p2.core.helpers.ServiceHelper;
import org.eclipse.equinox.p2.metadata.IArtifactKey;
@@ -49,8 +52,9 @@
                                 IArtifactKey key = new ArtifactKey("eclipse", "plugin", "org.eclipse.equinox.prov.engine", new Version("0.1.0.200709241631"));
                                 IArtifactDescriptor[] descriptors = repository.getArtifactDescriptors(key);
                                 assertTrue("Artifact Descriptor for engine missing", descriptors.length == 1);
-
-                                  new Optimizer(repository).run();
+                                  List optimizers = new ArrayList();
+                                  optimizers.add(new Pack200Optimizer());
+                                  new Optimizer(repository, optimizers).run();
                                 descriptors = repository.getArtifactDescriptors(key);
                                 assertTrue("Optimization was a no-op", descriptors.length == 2);





_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
 

_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev


Back to the top