Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [ptp-dev] Fwd: TSC and LML for Fujitsu PJM
Mars.2 RC4 is this week, so it's too late for this release. You could either aim for Neon in June or we could do an interim release of PTP. 
Greg

On Feb 12, 2016, at 3:32 AM, Wyatt Spear <wspear@xxxxxxxxxxxxxx> wrote:

Greetings,

Peter has some new LML support for the FX10 and K systems. What is our release schedule looking like at this point?

Thanks,
Wyatt

---------- Forwarded message ----------
From: Peter Bryzgalov <petr.bryzgalov@xxxxxxxx>
Date: Thu, Feb 11, 2016 at 7:14 PM
Subject: TSC and LML for Fujitsu PJM
To: Wyatt Spear <wspear@xxxxxxxxxxxxxx>


Hi Wyatt,

I would like to submit Target System Configurations and LML drivers for Fujitsu FX10 and "K" computers.
Do you know when the deadline is for the next Eclipse PTP release?

Best regards,
Peter Bryzgalov

RIKEN AICS
HPC Usability Research Team
Research Scientist

On 20 May, 2015, at 02:03, Wyatt Spear <wspear@xxxxxxxxxxxxxx> wrote:

I think the current release is feature and API locked at this point so it would show up in a smaller incremental release afterward.
=Wyatt

On Mon, May 18, 2015 at 11:54 PM, Peter Bryzgalov <pyotr777@xxxxxx> wrote:
Can I expect that it will be possible to merge trace files in new Eclipse version?
Anyway, I really appreciate your help!

Peter

On 19May, 2015, at 15:48, Wyatt Spear <wspear@xxxxxxxxxxxxxx> wrote:

Greetings,
I'm glad this is working for you now! I wasn't able to reproduce the problem over here. 
Trace files are very different from the profile files stored in the database. They are not compatible. There used to be an automatic workflow that would handle merging trace files but it was for an older version of the ETFw schema. I'll give some thought to handling it in the new version. Ideally it could be done by extending the JAXB workflow definition, but it might be a quicker process to just implement it in the Java side of things. Thanks for bringing this to my attention!

Regards,
Wyatt Spear

On Mon, May 18, 2015 at 11:39 PM, Peter Bryzgalov <pyotr777@xxxxxx> wrote:
Dear Wyatt,

I have reinstalled TAU on my local computer and now profiles are getting stored in my local DB, and I can access them from  Eclipse. Sorry for disturbing you with this problem.

May I ask you a couple of questions?

Though profiles (I tested with merged TAU profiles) are stored in DB, TAU traces are not. After tracing job finishes I get an error message "No profile data was located. Please make sure profile data is available".
Trace files I get are named like tautrace.N.0.0.trc and events.N.edf

- Can Eclipse store trace files in DB?

- Individual trace files need to be merged to be opened in Jumpshot. Is it possible to set up post-processing commands in Eclipse PTP? It would be nice to be able to define commands in TSC.

Kind regards,
Peter


On 2 May, 2015, at 03:46, Wyatt Spear <wspear@xxxxxxxxxxxxxx> wrote:

Greetings,
I believe Greg committed a fix to the issue with the environment variables propagating properly. Did you have a chance to try that out? Hopefully being able to manually set the profile output location will solve your problem but if not I can try to dig deeper. 

The DB provides a couple advantages, mostly to organization and ease of access to the profiles. Tau's standalone tools can use the database as well as eclipse and having them in the database enables some features, mostly related to comparing multiple trials, that cannot be done with files on the file-system.

Regards,
Wyatt

On Thu, Apr 30, 2015 at 11:30 PM, Peter Bryzgalov <pyotr777@xxxxxx> wrote:
Greetings Wyatt,
How are you?
Any news on profiles not getting into DB problem? 
What merits using DB has comparing to accessing profiles files through project explorer in Eclipse? 

Kind regards,
Peter


On 31 Mar, 2015, at 14:15, Peter Bryzgalov <pyotr777@xxxxxx> wrote:

Hello Wyatt,
Thank you for posting the bug in ppt-dev list.

I investigated a bit what is happening on login node. I attach a log of commands executed on login node. I get this log by executing ps ax in a loop, so it is possible that the log missed some commands. Only commands with different PIDs are in the log. 

In Eclipse in console window an error message is displayed:

Packed profile is here (login node):
/volume1/home/ra000007/a03106/C_MPI/Profiles/mpi-pdt-fujitsu/2015-03-31_13-10-27/C_MPI_mpi-pdt-fujitsu_2015-03-31_13-10-27.ppk

I have environment variables set like this on login node:
PROFILEDIR=/volume1/home/ra000007/a03106/C_MPI
TAUDIR=/volume1/home/ra000007/a03106/tau-2.23.1
TAU_THROTTLE=1
TAU_MAKEFILE=/volume1/home/ra000007/a03106/tau-2.23.1/sparc64fx/lib/Makefile.tau-mpi-pdt-fujitsu
TAU_VERBOSE=1
TAU_PROFILING=1

Kind regards,
Peter

<profiling on K login node (short).txt>

On 31 Mar, 2015, at 08:35, Wyatt Spear <wspear@xxxxxxxxxxxxxx> wrote:

I'm not sure what's actually going on here. Are you saying the ppk file is being generated successfully? Once you have the ppk file the --pack operation in the profile directory would be redundant. If I'm reading this right and it's trying to pack the profiles twice then something is very off in the profile management logic.
I posted about your environment export bug on the ptp-dev list.
=Wyatt

On Wed, Mar 25, 2015 at 10:10 PM, Peter Bryzgalov <pyotr777@xxxxxx> wrote:
Greetings Wyatt,
Thank you for your replay!

As far as I can understand, here is how it works now:
When job is executed on computing nodes profiles for each MPI rank generated in CWD.
After job has finished, individual profiles are staged out to login node in the job's top directory — /home/ra000007/a03106/C_MPI in my case.
After that individual profile files are packed into a .ppk file, which is put into Profiles subdir, for example:
/home/ra000007/a03106/C_MPI/Profiles/mpi-pdt-fujitsu/2015-03-26_10-56-05/C_MPI_mpi-pdt-fujitsu_2015-03-26_10-56-05.ppk
Individual profiles for each MPI rank are deleted.

After that paraprof is called with .ppk file name as a parameter:
paraprof --pack C_MPI_mpi-pdt-fujitsu_2015-03-26_10-56-05.ppk

But since this file is in Profiles subdirectory, paraprof cannot find it.

So I want the packed profile to be in the job's top directory. Configuring paraprof so that it searches for .ppk file in Profiles subdir would be also OK.

Setting PROFILEDIR in Environment tab of Profile Configurations has ho effect. I filed a bug here: https://bugs.eclipse.org/bugs/show_bug.cgi?id=463033

I also set PROFILEDIR variable in tau.bashrc which is called from .bashrc on every log in to login node. 


In .bashrc:
source ~/tau.bashrc
In tau.bashrc :
# echo "tau.bashrc"
export PKG=/volume1/home/ra000007/a03106
export TAUDIR=$PKG/tau-2.23.1
export TAU=$TAUDIR/sparc64fx/lib
export PATH=/opt/FJSVtclang/1.2.0/util/bin:$PATH
export LD_LIBRARY_PATH=$MPI_HOME/lib64:$PATH
export PATH=$TAUDIR/sparc64fx/bin:$PATH
export TAU_MAKEFILE="$TAU/Makefile.tau-mpi-pdt-fujitsu"

export TAU_SAMPLING="1"
export PROFILEDIR="/volume1/home/ra000007/a03106/C_MPI"
export TAU_PROFILING="1"
export TAU_THROTTLE="1"
export TAU_VERBOSE="1"
export TRACEDIR="/volume1/home/ra000007/a03106/C_MPI"

Kind regards,
Peter


On 26 Mar, 2015, at 07:22, Wyatt Spear <wspear@xxxxxxxxxxxxxx> wrote:

Greetings,
We have encountered some complications with this in the past. By default profiles should be generated where the application run which should be detected by the profile manager automatically. In theory setting the PROFILEDIR environment variable in the runtime settings tab of the TAU tool configuration should allow a different output/search directory to be specified manually but I suspect this feature has not been maintained. Could you let me know your specific requirements and I can see about adjusting/re-introducing this feature.

I believe ptp-dev gets the most attention from the PTP team while user is mostly for standard usage questions rather than discussion of issues or new features. If you are requesting a specific feature or reporting an issue I suggest putting it in bugzilla and then including a link to the bugzilla entry in your communication with the dev mailing list.

Regards,
Wyatt Spear

On Tue, Mar 24, 2015 at 8:23 PM, Peter Bryzgalov <pyotr777@xxxxxx> wrote:
Greetings Wyatt,

I would appreciate very much if you could comment on where to look for settings to change the path Eclipse uses for profiles.

By the way, is ptp-user mailing list the right place for posting questions about TSC development? Should I use Bugzilla or ptp-dev instead?

Kind regards,
Peter


> On 3 Mar, 2015, at 11:50, Peter Bryzgalov <pyotr777@xxxxxx> wrote:
>
> Dear Wyatt,
>
> I have question about profiling with Eclipse PTP and TAU.
>
> I run profiling job from Eclipse. When a job is finished one ParaProf profile can be found on remote system:
> /home/ra000007/a03106/C_MPI/Profiles/mpi-pdt-fujitsu/2015-03-02_17-09-41/C_MPI_mpi-pdt-fujitsu_2015-03-02_17-09-41.ppk
> In Eclipse I want to store profiles in a DB while also keeping profiles in FS (checkbox on Data Collection tab).
>
> After job is finished I get an error:
> Error executing remote commandMoving profile data to: ssh://K/volume1/home/ra000007/a03106/C_MPI/C_MPI_mpi-pdt-fujitsu_2015-03-02_17-21-25.ppk
> It seems Eclipse looks for a profile in a wrong place.
>
> How can I set up Eclipse to work with profiles?
>
> I found too little information here: http://wiki.eclipse.org/PTP/designs/perf_tools_framework/TAU#Profile_a_Project_with_TAU_and_the_PTP
>
> I'm sorry to bother you.
> Kind regards,
> Peter
>














_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev

Back to the top