[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [higgins-dev] Performance Instrumentation of IdAS source

So, you're saying that the dynamic tools aren't reliable because they don't subtract out the time spent in a call stack where we are performing cp-specific tasks, especially things like making ldap wire hits to gather identity data.  Is that correct?


I assumed these tools would be able to allow one to filter out / ignore time spent in certain routines, but I'm no expert, I haven't used them.


If hand-instrumenting our code is the only option, then so be it I  guess.  I would at least not do this:

               final String METHODNAME = " _emitXDASRecord ";

since method names change and it's a pain to update these kinds of reference.  Instead, I'd do this:

                if( _log.isTraceEnabled( ) ){

                        sw.stop();

                        _log.trace(EXIT+MSGSEP+TIMESPENTIN+PerfUtil.getMethod()+" [ " + sw + " ] ");

                }

where there's some PerfUtil class with a method like:

public static String getMethod() {

   return new Exception().getStackTrace()[1].getMethodName()

}


of if u don't want the "new Exception()" hit every time, set up a variable once where you start your stopwatch.


Jim


>>> Prakash C Mallick <prakash.mallick@xxxxxxxxxx> 09/15/08 7:23 AM >>>

Hi Jim,
           I just did a brief analysis of TPTP and the performance measuring feature supported by TPTP suite of products. It seems Profiling Tool supported by TPTP uses JVM based profiling to calculate the time and memory consumed by an application/component/methods. I am still not sure how it will be helpfull if IdAS Packages are used in production environment in different execution contexts.

But if you see my proposal intially I posted where I was not in favour of using dynamic instrumentation(JVM based profiling), as it doesn't produce any reliable performance data. Please relook into the proposal below and suggest me how we should move on this.


Thanks and Regards,
Prakash
---------------------------------------------------------------------------------------------------
Advisory Software Engineer,
Tivoli Security Products
ETZ, Plot No. 3 , Rajiv Gandhi Infotech Park
Phase II, Hinjewadi, Pune - 411057
Ph: +91-20-42025544
e-mail: prakash.mallick@xxxxxxxxxx
---------------------------------------------------------------------------------------------------



"Jim Sermersheim" <jimse@xxxxxxxxxx> 
Sent by: higgins-dev-bounces@xxxxxxxxxxx

09/11/2008 10:49 PM

Please respond to
"Higgins \(Trust Framework\) Project developer discussions"        <higgins-dev@xxxxxxxxxxx>


To

<higgins-dev@xxxxxxxxxxx>

cc


Subject

Re: [higgins-dev] Performance Instrumentation of IdAS source








This is exciting.  Can u send a code snippet of how you do this?  Also, is there an accompanying parser which can graph the results?  If not, no biggie, this is better than nothing.
Do you think we should investigate using TPTP for doing this?  Here's a link http://www.eclipse.org/tptp/.I assume it does it without requiring one to hand-tool their code.

Jim 

>>> Prakash C Mallick <prakash.mallick@xxxxxxxxxx> 09/11/08 8:03 AM >>>

Hi Jim and All,
         If this is fine with all of you I would be fine to contribute to performance instrumentation implementation without affecting the production cycle. In fact I have already instrumented JNDI-CP using StopWatch classes for my prototype  implementation which shows the following output when perfermance instrumentation is enabled in Trace mode.


Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext _setupContext
FINEST: -->> ENTERING
Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext _setupContext
FINE: Attempting to create initial context: ldap://ltoyota.in.ibm.com:3389
Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext _setupContext
FINEST: <<-- EXITING
---- time spent in  _setupContext()  [ 0:00:00.015 ]
Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext _emitXDASRecord
FINEST: -->> ENTERING
Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext _emitXDASRecord
FINEST: <<-- EXITING
---- time spent in  _emitXDASRecord  [ 0:00:00.000 ]
Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext open
FINE: Context opened:  as: cn=root
Aug 29, 2008 6:07:22 PM org.eclipse.higgins.idas.cp.jndi.JNDIContext open
FINEST: <<-- EXITING
---- time spent in  open(Object )  [ 0:00:00.015 ]

We are yet to finalize the format of of permance data marked red.


Please let me know if above implementation looks ok, so that I can apply the same implementation in other IdAS packages.


Thanks and Regards,
Prakash
---------------------------------------------------------------------------------------------------
Advisory Software Engineer,
Tivoli Security Products
ETZ, Plot No. 3 , Rajiv Gandhi Infotech Park
Phase II, Hinjewadi, Pune - 411057
Ph: +91-20-42025544
e-mail: prakash.mallick@xxxxxxxxxx
---------------------------------------------------------------------------------------------------



"Jim Sermersheim" <jimse@xxxxxxxxxx>
Sent by: higgins-dev-bounces@xxxxxxxxxxx
 
08/19/2008 10:12 PM

Please respond to
"Higgins \(Trust Framework\) Project developer discussions"        <higgins-dev@xxxxxxxxxxx>



To

<higgins-dev@xxxxxxxxxxx>

cc


Subject

Re: [higgins-dev] Performance Instrumentation of IdAS source 











We can either instrument, or we can use the inmem CP.  In fact, I've been trying to migrate the SPI code to a point where the inmem cp is pretty much obsolete.  Still, I'm not at all opposed to instrumenting idas.cp.spi and idas.common as long as the instrumentation doesn't eat cycles in production environments.


Getting some baseline performance tests sounds great to me -- also it would be nice if someone could do some scalability testing.  There are places in the spi that we're certain have scalability problems.


Jim


>>> Prakash C Mallick <prakash.mallick@xxxxxxxxxx> 08/19/08 8:00 AM >>>


Hi All,
         We are not very sure if there is any plan or any mechanim already used to measure the performance of IdAS components. As in almost all usage performance is the mandatory non-feature requiement that  is supported by all products, we need to provide a mechanism to  mesure the performance of IdAS components(APIs) and also if possible we can provide performance bench mark for the same.

Requirement (To measure the performance of IdAS APIs)
-------------------------------------------------------------------------------------
Since  IdAS components are inteded to be used along with consumer code and as well as with service provider codes, its always a good idea to isolate the time consumed by IdAS APIs only when used along with these codes and let the consumer and service provider decide the performance impact of the code owned by them.
At present its not possible to identify performance issues that occurs becuase of the following reasons as we have seen so far  through debugging,
      1. Slow response in accessing  remote resources such as context dbs,remote objects ,operating on remote APIs and so on becuase bandwidth or any other infrastructure problem.
       2. Performance issue becuase of error prone or poorly written SPI code or consumer code.

Proposal to measure performance of IdAS code
-----------------------------------------------------------------------
We can use one of the following options to measure the performance

1.  Measuirng performanceof IdAS APIs  by using dynamic instrumentation(using various JVM profiling options during run time) .
2. Implementing static instrumentation(by adding  least CPU bound time based APIs such as start,stop as used in StopWatch kind of implementation to calculate the time spent by various APIs).
3. Possibly by building/executing IdAS based application in some platform provided by performance measuring tools .

But as mentioned above IdAS components are not used in isolation like other stand-alone application, so its not possible to get more accurate or realistic time data as consumed by these APIs to perform a task in a production environment  by using either dynamic instrumentation mechanism or any performance tools.

So using static instrumentation we will have more control to extract the  time consumed by IdAS APIs from external codes such as consumer code and service provider code. Once the time consumed data is obtained through instumentation , the same can be logged (using trace/debug log level) to a log file.This mechanism will also add entry and exit time into a method which is or can be added by apache common log calls already used in IdAS source codes. This  instrumentation can be protected through IsDebuggable/IsLoggable calls so that we will enable it at a particular log level such as TRACE/DEBUG .
Overally this feature will help the cosumer or service provider stakeholders to know the performance of IdAS code in their environment. In addition to that we can create a common package such as org.eclipse.higgins.util.performance to provide utility  methods for static instrumentation which can be used by IdAS developer,consumers and Service Providers.
We have a mechanism as suggested by David and TIM Hahn to implment the same using Stopwatch similar to the one provided by apache ..Please refer the link  http://commons.apache.org/lang/api/org/apache/commons/lang/time/StopWatch.html for more information.
If all of you think we need  this support , please let me know. Please provide your inputs on this. I am planning to have a prototype for this once I get some feed backs from all of you.

Thanks and Regards,
Prakash
---------------------------------------------------------------------------------------------------
Advisory Software Engineer,
Tivoli Security Products
ETZ, Plot No. 3 , Rajiv Gandhi Infotech Park
Phase II, Hinjewadi, Pune - 411057
Ph: +91-20-42025544
e-mail: prakash.mallick@xxxxxxxxxx
---------------------------------------------------------------------------------------------------
_______________________________________________
higgins-dev mailing list
higgins-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/higgins-dev

_______________________________________________
higgins-dev mailing list
higgins-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/higgins-dev