Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [tracecompass-dev] Beginner questions for tracecompass (navigation, filtering, working with large datasets)
  • From: Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
  • Date: Wed, 23 Feb 2022 15:38:14 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wYM6X4vx2SaFrUeFEwa6fx+JA1rmOaR3PsdYvUiOPkI=; b=MQ1XUGPaq+qRO+A+IWKoHBcUKz/OSyjyGNoPttfSf/Shh2+J4lseZ3uJ6jkoEijrHbgx69xXqxtKojjnQJ6WFgjwoFjmydKlj2LAttfJbN5tImgCMep9gVWjNLWObpD5F70tNakYs2AtCB+zyzLod5i50hQd+OMGfmAMXTYTLVKd4onazZuvxy82AMP9lPWjKIeOgn1KXfzGCDQxdhGib2Uf6V7mLZgwpWQh66ZtIrE8/Z67fnFPvPtkUZgplrQHqU/m/urXdmGGM58vte6D/vbIcBAcyNgs/QXAJ680HaahLTkgzL8lMtFJsRQmPHQnc913IUYVi2BX0tdwBmDdbQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XI98OPOD1k2W59Bd/p5zEpKZW9x0RiqDzQ+N1ZVjn5ai/YWoAPgkRs8PmMKraiJzd0qOU8z1X6VTf1lkPQYgX//e+s2LYjg3ZPuEVvGRNG2hsySruLlqx/KqHYoPIqiq+lx60oJXDoHg7uO19LjKP9BE9SXtw8QRwk7fS69jx0yxk/rxAlVmUS0PXolHIwAOfc1F6rUv5AS2MyVfV8UPrddZaXkpJHBIoupIp/lKBavRKL598WjJMH0pBQoHGUVEruhxNein8Q9RbVMksrTBrPj+gZmhtxcNnNo5TGp8CulrbS5rjFOxkqMNLsUsR2FD4MWdrKOQCeKXZPSwSxwSEQ==
  • Delivered-to: tracecompass-dev@xxxxxxxxxxx
  • List-archive: <https://www.eclipse.org/mailman/private/tracecompass-dev/>
  • List-help: <mailto:tracecompass-dev-request@eclipse.org?subject=help>
  • List-subscribe: <https://www.eclipse.org/mailman/listinfo/tracecompass-dev>, <mailto:tracecompass-dev-request@eclipse.org?subject=subscribe>
  • List-unsubscribe: <https://www.eclipse.org/mailman/options/tracecompass-dev>, <mailto:tracecompass-dev-request@eclipse.org?subject=unsubscribe>
  • Msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=e1b36dc6-036f-4818-96dd-e7314bb8ce21; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2022-02-16T09:04:50Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
  • Suggested_attachment_session_id: d721cf64-34ad-db78-7f6f-539aaded8904
  • Thread-index: AdgjFFM+yk8LgKseRO+Zxf80PqTP5wAGjNXOAWck3qQ=
  • Thread-topic: Beginner questions for tracecompass (navigation, filtering, working with large datasets)
Aprameya,

I took the liberty of making a youtube video for how to navigate in trace compass. https://youtu.be/lE6c2Vew_B4 <-- hope it helps!

From: Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
Sent: Wednesday, February 16, 2022 7:38 AM
To: tracecompass-dev@xxxxxxxxxxx <tracecompass-dev@xxxxxxxxxxx>
Cc: Aprameya Rao <Aprameya.Rao@xxxxxxxxxxxxx>
Subject: Re: Beginner questions for tracecompass (navigation, filtering, working with large datasets)
 

Hi Aprameya, thanks for the interest!


> 1. How do I navigate to a particular timestamp? Lets say my traces are spread over two hours from 02:00:00 to 04:00:00. How do I tell the events view to go to a specific timestamp (say 03:30:00) and start rendering traces from there?


There are many ways, I will list some:

  • Use the mouse to navigate (right click drag)
  • Go to the histogram view and manually enter the start time
  • Use the keyboard to navigate (W - A - S - D + mouse)
  • Import only the traces that are spanning the relevant time using the import wizard.



> 2. It seems that applying a filter starts from the very first event as opposed to the timestamp of the current event and downward. Is this correct? If so, how can I make it start from the current time and go down as opposed to moving the cursor to the start of the experiment and then applying the filter?


Are you talking about the events table?  Yes, the filter starts at the top and works its way down. The reason being that we want to maintain the event ranks. If you want to start two threads, one at the cursor and one at the top, patches are really welcome. None of the developing partners or stakeholders have expressed interest in that functionality, but it sounds really cool!


> 3. Are there tips and tricks for working with large amounts of traces? Are there specific aspects of the CTF format that if implemented help tracecompass do better?


This is a difficult and fun problem since you are dealing with two issues: IO and CPU. If the events are too compressed, you use more CPU, less IO, if the events are too large, CPU is less used. I suspect NVMe technology changed the "sweet spot", but AMD making the CPU market competitive is also moving the needle, who knows, maybe we're back at the same equilibrium point. The important part: more tracing is required.

This is general, I cannot think of a viewer that wouldn't benefit from this.

  • Align your event starts on bytes, not bits
  • If possible, align your packet sizes on the "disk block size" so if a node is 4096 bytes, make each packet size a multiple of 4096
  • Strings are not your friend, use integers for timestamps and whatnot
  • Try stuff, we show you the reading rate in events/second, you can benchmark your changes.
I have a multi-threaded parser patch here: https://git.eclipse.org/r/c/tracecompass/org.eclipse.tracecompass/+/54825 Also, the dorsal (www.dorsal.polymtl.ca) lab is working on scaling trace compass to 10k traces at once. You can contact them for more details, I suspect that one will jump in to this thread though. 😉


>4. I notice that lttng creates an index file for some of its traces. Does tracecompass use this file?


We rebuild those index files since they are not in the CTF spec. I have made a patch to support it, but honestly, there was no demand for this feature. https://git.eclipse.org/r/c/tracecompass/org.eclipse.tracecompass/+/39247

I hope this helps!
cheers!

Matthew
From: tracecompass-dev <tracecompass-dev-bounces@xxxxxxxxxxx> on behalf of Aprameya Rao via tracecompass-dev <tracecompass-dev@xxxxxxxxxxx>
Sent: Wednesday, February 16, 2022 4:05 AM
To: tracecompass-dev@xxxxxxxxxxx <tracecompass-dev@xxxxxxxxxxx>
Cc: Aprameya Rao <Aprameya.Rao@xxxxxxxxxxxxx>
Subject: [tracecompass-dev] Beginner questions for tracecompass (navigation, filtering, working with large datasets)
 

Hi

 

I’m just getting started with tracecompass and large files. I have opened up a CTF trace with around 60 million events spanning about two hours and 5-6 GiB and have some basic questions. My traces are stored in multiple files in different folders and I’ve added them all to a project. I’ve also created an experiment covering all traces.

 

1. How do I navigate to a particular timestamp? Lets say my traces are spread over two hours from 02:00:00 to 04:00:00. How do I tell the events view to go to a specific timestamp (say 03:30:00) and start rendering traces from there?

2. It seems that applying a filter starts from the very first event as opposed to the timestamp of the current event and downward. Is this correct? If so, how can I make it start from the current time and go down as opposed to moving the cursor to the start of the experiment and then applying the filter?

3. Are there tips and tricks for working with large amounts of traces? Are there specific aspects of the CTF format that if implemented help tracecompass do better?

4. I notice that lttng creates an index file for some of its traces. Does tracecompass use this file?

 

Thanks

Aprameya

 

Back to the top