Otter OMPT

Otter OMPT is an OMPT plugin for tracing & visualising the structure of loop- & task-based OpenMP programs, allowing HPC developers to see the structure of their OpenMP 5.x programs from the perspective of the OpenMP runtime, without any modification of the target application. Otter uses the OpenMP Tools interface in OpenMP 5.0 to observe OpenMP runtime events, extracting from this data the structure of the target application independent of the particular scheduling of tasks at runtime.

Features

Trace the task creation and synchronisation constructs of an OpenMP 5.x program without any modification of the source - no need to add any instrumentation to the target application.
Supports synchronisation due to taskwait and taskgroup constructs.
Supports nested tasks and nested parallelism.
No additional thread synchronisation - won’t accidentally serialise the target application.

Example

Take this example code which uses nested tasks synchronised by a taskwait barrier to calculate the nth Fibonacci number:

int fibonacci(int n) {
    int i, j;
    if (n<2) return n;
    #pragma omp task shared(i) firstprivate(n)
        i = f(n-1);
    #pragma omp task shared(j) firstprivate(n)
        j = f(n-2);
    #pragma omp taskwait
    return i+j;
}

We can speculate about the structure of this code in terms of tasks and their synchronisation, but how can we check that our ideas match reality? This is a challenge even for the simple code above, and soon becomes impossible for complex task-based code. We might try using performance analysis tools to trace or profile an application, providing a thread-centric view of a specific arrangement of tasks. While this gives us insight into the application’s runtime performance, we would still struggle to get a clear picture of the application’s overall structure. Using Otter we can observe the true structure of a task-based OpenMP application, all without modifying the application’s source.

Getting Started

See the installation page for building & installing Otter.

Tracing a target OpenMP application omp-demo is as simple as:

OMP_TOOL_LIBRARIES=<otter-install-prefix>/lib/libotter-ompt.so ./omp-demo

If everything is set up correctly, you should see output like this (depending on the specific OpenMP runtime you are using):

Intel(R) OMP version: 5.0.20210428, OMP v. 201611
Otter was compiled with icc (ICC) 2021.3.0 20210609
Starting OTTer...
Trace output path:             trace/otter_trace.[pid]

Registering callbacks:
ompt_callback_thread_begin       | ompt_set_always (5)
ompt_callback_thread_end         | ompt_set_always (5)
ompt_callback_parallel_begin     | ompt_set_always (5)
ompt_callback_parallel_end       | ompt_set_always (5)
ompt_callback_task_create        | ompt_set_always (5)
ompt_callback_task_schedule      | ompt_set_always (5)
ompt_callback_implicit_task      | ompt_set_always (5)
ompt_callback_work               | ompt_set_always (5)
ompt_callback_masked             | ompt_set_always (5)
ompt_callback_sync_region        | ompt_set_always (5)

PROCESS RESOURCE USAGE:
       maximum resident set size:    11916 kb
page reclaims (soft page faults):     5644
  page faults (hard page faults):        1
          block input operations:        0
         block output operations:       72

                         threads:        3
                parallel regions:        2
                           tasks:       25
OTTER_TRACE_FOLDER=/home/adam/otter/trace/otter_trace.[pid]

By default, Otter writes a trace to trace/otter_trace.[pid] - the location and name of the trace can be set with the OTTER_TRACE_PATH and OTTER_TRACE_NAME environment variables.