| Re: [tracecompass-dev] Help regarding arrows in EASE scripting |
|
Hi Lea,
thank you very much in the interest in Trace Compass. See below for my answers.
Regards,
Bernd
From: tracecompass-dev <tracecompass-dev-bounces@xxxxxxxxxxx> on behalf of Lea.Jungmann@xxxxxx <Lea.Jungmann@xxxxxx>
Sent: June 23, 2023 9:31 AM To: tracecompass-dev@xxxxxxxxxxx <tracecompass-dev@xxxxxxxxxxx> Subject: [tracecompass-dev] Help regarding arrows in EASE scripting Hi!
So I’ve been trying to use python with EASE scripting to make custom TimeGraphs but I run into two problems:
The first, minor, one is that every time I change the script, I have to change the name of the Analysis to make the changes appear. Is there a way around that?
[Bernd Hufmann] The state system should be always re-created due to passing False to "analysis.getStateSystem(False)". Having said that, that works well in Linux but
in case you're running in Windows, there might be an issue because the state system file is open and in Windows you can't overwrite a file if it's in use (which it is). So, you can delete the supplementary files : https://archive.eclipse.org/tracecompass/doc/org.eclipse.tracecompass.doc.user/Trace-Compass-Main-Features.html#Deleting_Supplementary_Files.
Maybe just select the "script" entry which will delete all state systems created by any script for that trace, the other state systems or index file don't need to be deleted.
My second issue is that even though I create arrows and hand them over to the TimeGraphProvider, they do not appear. I have been heavily relying on the script given in the tutorial here [1]. [Bernd Hufmann] I don't see anything obvious wrong. I converted the tutorial example [1] written in _javascript_ to Python and the arrows are created. I used some code from your example, and it helped me a lot. See below the Python script I'm using:
----------------------------------------------------------
# The MIT License (MIT)
#
# Copyright (C) 2019 - Geneviève Bastien <gbastien@xxxxxxxxxxxx>
# Copyright (C) 2019 - École Polytechnique de Montréal
# Copyright (c) 2023 - Ericsson
#
# Load the proper modules
loadModule("/TraceCompass/Trace")
loadModule("/TraceCompass/Analysis")
loadModule("/TraceCompass/DataProvider")
loadModule("/TraceCompass/View")
loadModule('/TraceCompass/Utils')
from py4j.java_gateway
import JavaClass
import time
# Get the active trace
trace = getActiveTrace()
# Get an event iterator on that trace
iter = getEventIterator(trace)
# Associate a TID with an mpi worker
tidToWorkerMap = {}
# Get an analysis
analysis = createScriptedAnalysis(trace,
"ringTimeLine.py")
if analysis
is None:
print("Trace is null")
exit()
def strToVarargs(str):
object_class = java.lang.String
object_array = gateway.new_array(object_class,
1)
object_array[0]
= str
return object_array
def numberToVarargs(num):
object_class = java.lang.String
object_array = gateway.new_array(object_class,
1)
object_array[0]
= str(num)
return object_array
# Get the analysis's state system so we can fill it, false indicates to create a new state system even if one already exists
ss = analysis.getStateSystem(False)
# Save information on the pending arrows
pendingArrows = {}
# Variable to save the arrow information
arrows = []
# Iterate through the events
event =
None
while iter.hasNext():
event = iter.next()
eventName = event.getName()
if eventName ==
"ring:init":
tid = getEventFieldValue(event,
"context._vtid")
worker_id = getEventFieldValue(event,
"worker_id")
tidToWorkerMap[tid] = worker_id
elif eventName ==
"ring:recv_entry":
tid = getEventFieldValue(event,
"context._vtid")
worker_id = tidToWorkerMap[tid]
#Save the state of the resource as waiting for reception
quark = ss.getQuarkAbsoluteAndAdd(numberToVarargs(worker_id))
ss.modifyAttribute(event.getTimestamp().toNanos(),
"Waiting for reception", quark)
elif eventName ==
"ring:recv_exit":
tid = getEventFieldValue(event,
"context._vtid")
worker_id = tidToWorkerMap[tid]
source = getEventFieldValue(event,
"source")
# Remove the waiting for reception state
quark = ss.getQuarkAbsoluteAndAdd(numberToVarargs(worker_id));
ss.removeAttribute(event.getTimestamp().toNanos(), quark);
# Complete an arrow if the start was available
pending = pendingArrows[worker_id];
if not (pending
is None):
# There is a pending arrow (ie send) for this message
pendingArrows[worker_id] =
None;
pending["endTime"]
= event.getTimestamp().toNanos();
arrows.append(pending);
elif eventName ==
"ring:send_entry":
tid = getEventFieldValue(event,
"context._vtid")
worker_id = tidToWorkerMap[tid]
dest = getEventFieldValue(event,
"dest")
# Save the state of the resource as sending
quark = ss.getQuarkAbsoluteAndAdd(numberToVarargs(worker_id))
ss.modifyAttribute(event.getTimestamp().toNanos(),
"Sending", quark)
# Save a pending arrow
pendingArrows[dest] = {"time" :
event.getTimestamp().toNanos(), "source" : worker_id,
"dest" : dest}
elif eventName ==
"ring:send_exit":
tid = getEventFieldValue(event,
"context._vtid")
worker_id = tidToWorkerMap[tid]
# Remove the sending for reception state
quark = ss.getQuarkAbsoluteAndAdd(numberToVarargs(worker_id))
ss.removeAttribute(event.getTimestamp().toNanos(), quark)
# Done parsing the events, close the state system at the time of the last event, it needs to be done manually otherwise the state system will still be waiting
for values and will not be considered finished building
if (not (event
is None)):
ss.closeHistory(event.getTimestamp().toNanos())
# Get list wrappers from Trace Compass for the entries and arrows. The conversion between _javascript_ list and java list is not direct, so we need a wrapper
tgEntries = createListWrapper()
tgArrows = createListWrapper()
# /* Prepare the time graph data, there is few enough entries and arrows that it can be done once and returned once */
# Map the worker ID to an entry ID
mpiWorkerToId = {}
# Get all the quarks of the entries
quarks = ss.getQuarks(-1,
strToVarargs("*"))
# Prepare the entries
mpiEntries = []
for quark
in quarks:
# Get the mpi worker ID, and find its quark
mpiWorkerId = ss.getAttributeName(quark)
# Create an entry with the worker ID as name and the quark. The quark will be used to populate the entry's data.
entry = createEntry(mpiWorkerId, {'quark' :
quark})
mpiWorkerToId[mpiWorkerId] = entry.getId()
mpiEntries.append(entry)
# Sort the entries numerically
#mpiEntries.sort(function(a,b){return Number(a.getName()) - Number(b.getName())})
# Add the entries to the entry list
for entry
in mpiEntries:
tgEntries.getList().add(entry)
# Prepare the arrows
for arrow
in arrows:
# For each arrow, we get the source and destination entry ID from its mpi worker ID
srcId = mpiWorkerToId[str(arrow["source"])]
dstId = mpiWorkerToId[str(arrow["dest"])]
# Get the start time and calculate the duration
startTime = arrow["time"]
duration = arrow["endTime"]
- startTime
# Add the arrow to the arrows list
tgArrows.getList().add(createArrow(srcId, dstId, startTime, duration,
1))
# A function used to return the entries to the data provider. It receives the filter in parameter, which contains the requested time range and any additional
information
class EntryFunction(object):
def apply(self, parameters):
return tgEntries.getList()
class Java:
implements = ['java.util.function.Function']
# A function used to return the arrows to the data provider. It receives the filter in parameter, which contains the requested time range and any additional
information
class ArrowFunction(object):
def apply(self, parameters):
return tgArrows.getList()
class Java:
implements = ['java.util.function.Function']
entryFunction = EntryFunction()
arrowFunction = ArrowFunction()
# Create a scripted data provider for this analysis, using script functions to get the entries, row model data and arrows. Since the entries have a quark associated
with them which contains the data to display, there is no need for a scripted getRowData function, so we send null
provider = createScriptedTimeGraphProvider(analysis, entryFunction,
None, arrowFunction)
if not (provider
is None):
# Open a time graph view displaying this provider
openTimeGraphView(provider)
monitor = getScriptEngine().getMonitor()
while not monitor.isCanceled():
time.sleep(5)
----------------------------------------------------------
Here is the script I’ve been using:
loadModule('/TraceCompass/Analysis'); loadModule('/TraceCompass/Trace'); loadModule('/TraceCompass/View'); loadModule('/TraceCompass/DataProvider'); loadModule('/TraceCompass/Utils');
from py4j.java_gateway import JavaClass import time
trace = getActiveTrace() analysis = createScriptedAnalysis(trace, "Tasking")
if analysis is None: print("Trace is null") exit() ss = analysis.getStateSystem(False)
def strToVarargs(str): object_class = java.lang.String object_array = gateway.new_array(object_class, 1) object_array[0] = str return object_array
tgArrows = createListWrapper(); tgEntries = createListWrapper();
def runAnalysis(): iter = analysis.getEventIterator() last = None lastTime = 0
event = None while iter.hasNext(): if not(event is None): gateway.detach(event)
event = iter.next();
# Custom events if event.getName() == "pushed_channel": name = getEventFieldValue(event, "channel") if (not(name is None)): quark = ss.getQuarkAbsoluteAndAdd(strToVarargs(str(name))) ss.modifyAttribute(event.getTimestamp().toNanos(), "Push", quark)
if event.getName() == "unpushed_channel": name = getEventFieldValue(event, "channel") if (not(name is None)): quark = ss.getQuarkAbsoluteAndAdd(strToVarargs(str(name))) ss.removeAttribute(event.getTimestamp().toNanos(), quark)
if event.getName() == "activated_task": name = getEventFieldValue(event, "task") if (not(name is None)): quark = ss.getQuarkAbsoluteAndAdd(strToVarargs(str(name))) ss.modifyAttribute(event.getTimestamp().toNanos(), "Activated", quark)
if event.getName() == "start_executing_task": name = getEventFieldValue(event, "task") if(not(name is None)): quark = ss.getQuarkAbsoluteAndAdd(strToVarargs(str(name))) ss.modifyAttribute(event.getTimestamp().toNanos(), "Executing", quark)
if event.getName() == "stop_executing_task": name = getEventFieldValue(event, "task") if(not(name is None)): quark = ss.getQuarkAbsoluteAndAdd(strToVarargs(str(name))) ss.removeAttribute(event.getTimestamp().toNanos(), quark) last = quark; lastTime = event.getTimestamp().toNanos(); if event.getName() == "sched_switch": name = getEventFieldValue(event, "next_comm") if(not(name is None) and not(last is None)): quark = ss.getQuarkAbsoluteAndAdd(strToVarargs(str(name))) ss.removeAttribute(event.getTimestamp().toNanos(), quark) duration = lastTime - event.getTimestamp().toNanos();
# Arrows created and added to list here tgArrows.getList().add(createArrow(last, quark, lastTime, duration, 1))
if not(event is None): ss.closeHistory(event.getTimestamp().toNanos()) if not(ss.waitUntilBuilt(0)): runAnalysis()
quarks = ss.getQuarks(strToVarargs("*")) entries = []
#get all entries for quark in quarks: entries.append(ss.getAttributeName(quark))
# put all entries into wrapper list for entry in entries: quark = ss.getQuarkAbsolute(strToVarargs(str(entry))) listEntry = createEntry(entry, {"quark" : quark}) tgEntries.getList().add(listEntry) class EntryFunction(object): def apply(self, parameters): return tgEntries.getList();
class Java: implements = ['java.util.function.Function']
class ArrowFunction(object): def apply(self, parameters): return tgArrows.getList();
class Java: implements = ['java.util.function.Function']
entryFunction = EntryFunction() arrowFunction = ArrowFunction() provider = createScriptedTimeGraphProvider(analysis, entryFunction, None, arrowFunction); if not(provider is None): openTimeGraphView(provider) monitor = getScriptEngine().getMonitor() while not monitor.isCanceled(): time.sleep(5)
Is there something obvious, that I missed, that causes the arrows not to appear in the TimeGraph?
Best regards, Lea
|
