linux-uconsole/kernel/trace
Steven Rostedt (VMware) a1de406751 ring-buffer: Zero out time extend if it is nested and not absolute
commit 097350d1c6 upstream.

Currently the ring buffer makes events that happen in interrupts that preempt
another event have a delta of zero. (Hopefully we can change this soon). But
this is to deal with the races of updating a global counter with lockless
and nesting functions updating deltas.

With the addition of absolute time stamps, the time extend didn't follow
this rule. A time extend can happen if two events happen longer than 2^27
nanoseconds appart, as the delta time field in each event is only 27 bits.
If that happens, then a time extend is injected with 2^59 bits of
nanoseconds to use (18 years). But if the 2^27 nanoseconds happen between
two events, and as it is writing the event, an interrupt triggers, it will
see the 2^27 difference as well and inject a time extend of its own. But a
recent change made the time extend logic not take into account the nesting,
and this can cause two time extend deltas to happen moving the time stamp
much further ahead than the current time. This gets all reset when the ring
buffer moves to the next page, but that can cause time to appear to go
backwards.

This was observed in a trace-cmd recording, and since the data is saved in a
file, with trace-cmd report --debug, it was possible to see that this indeed
did happen!

  bash-52501   110d... 81778.908247: sched_switch:         bash:52501 [120] S ==> swapper/110:0 [120] [12770284:0x2e8:64]
  <idle>-0     110d... 81778.908757: sched_switch:         swapper/110:0 [120] R ==> bash:52501 [120] [509947:0x32c:64]
 TIME EXTEND: delta:306454770 length:0
  bash-52501   110.... 81779.215212: sched_swap_numa:      src_pid=52501 src_tgid=52388 src_ngid=52501 src_cpu=110 src_nid=2 dst_pid=52509 dst_tgid=52388 dst_ngid=52501 dst_cpu=49 dst_nid=1 [0:0x378:48]
 TIME EXTEND: delta:306458165 length:0
  bash-52501   110dNh. 81779.521670: sched_wakeup:         migration/110:565 [0] success=1 CPU:110 [0:0x3b4:40]

and at the next page, caused the time to go backwards:

  bash-52504   110d... 81779.685411: sched_switch:         bash:52504 [120] S ==> swapper/110:0 [120] [8347057:0xfb4:64]
CPU:110 [SUBBUFFER START] [81779379165886:0x1320000]
  <idle>-0     110dN.. 81779.379166: sched_wakeup:         bash:52504 [120] success=1 CPU:110 [0:0x10:40]
  <idle>-0     110d... 81779.379167: sched_switch:         swapper/110:0 [120] R ==> bash:52504 [120] [1168:0x3c:64]

Link: https://lkml.kernel.org/r/20200622151815.345d1bf5@oasis.local.home

Cc: Ingo Molnar <mingo@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Tom Zanussi <zanussi@kernel.org>
Cc: stable@vger.kernel.org
Fixes: dc4e2801d4 ("ring-buffer: Redefine the unimplemented RINGBUF_TYPE_TIME_STAMP")
Reported-by: Julia Lawall <julia.lawall@inria.fr>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-06-30 23:17:17 -04:00
..
blktrace.c blktrace: break out of blktrace setup on concurrent calls 2020-06-30 23:17:15 -04:00
bpf_trace.c bpf: fix nested bpf tracepoints with per-cpu data 2019-07-03 13:14:48 +02:00
ftrace.c ftrace: fpid_next() should increase position index 2020-02-24 08:34:52 +01:00
Kconfig Stop the ad-hoc games with -Wno-maybe-initialized 2020-05-20 08:18:45 +02:00
Makefile tracing: Allow gcov profiling on only ftrace subsystem 2018-08-21 09:11:49 -04:00
power-traces.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
preemptirq_delay_test.c tracing: Use trace_clock_local() for looping in preemptirq_delay_test.c 2018-10-17 15:35:33 -04:00
ring_buffer.c ring-buffer: Zero out time extend if it is nested and not absolute 2020-06-30 23:17:17 -04:00
ring_buffer_benchmark.c tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
rpm-traces.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
trace.c tracing: Add a vmalloc_sync_mappings() for safe measure 2020-05-14 07:57:20 +02:00
trace.h ftrace: Protect ftrace_graph_hash with ftrace_sync 2020-02-11 04:34:05 -08:00
trace_benchmark.c rcu: Rename cond_resched_rcu_qs() to cond_resched_tasks_rcu_qs() 2018-05-15 10:27:29 -07:00
trace_benchmark.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00
trace_branch.c Revert "x86/uaccess, ftrace: Fix ftrace_likely_update() vs. SMAP" 2019-07-03 13:14:41 +02:00
trace_clock.c tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
trace_entries.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00
trace_event_perf.c tracing: Fix race in perf_trace_buf initialization 2019-10-29 09:20:03 +01:00
trace_events.c tracing: Fix lock inversion in trace_event_enable_tgid_record() 2020-01-09 10:19:02 +01:00
trace_events_filter.c tracing: Avoid memory leak in process_system_preds() 2020-01-09 10:19:02 +01:00
trace_events_filter_test.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00
trace_events_hist.c tracing: Fix histogram code when expression has same var as value 2020-01-29 16:43:23 +01:00
trace_events_trigger.c tracing: Fix event trigger to accept redundant spaces 2020-06-30 23:17:17 -04:00
trace_export.c tracing: Add trigger file for trace_markers tracefs/ftrace/print 2018-05-29 08:28:53 -04:00
trace_functions.c tracing: Update stack trace skipping for ORC unwinder 2018-01-23 15:57:00 -05:00
trace_functions_graph.c tracing/fgraph: Fix set_graph_function from showing interrupts 2018-12-08 12:59:07 +01:00
trace_hwlat.c tracing/hwlat: Don't ignore outer-loop duration when calculating max_latency 2019-10-17 13:45:39 -07:00
trace_irqsoff.c tracing/fgraph: Fix set_graph_function from showing interrupts 2018-12-08 12:59:07 +01:00
trace_kdb.c tracing: Silence GCC 9 array bounds warning 2019-06-25 11:35:52 +08:00
trace_kprobe.c tracing/kprobes: Fix a double initialization typo 2020-05-14 07:57:16 +02:00
trace_kprobe_selftest.c selftest/ftrace: Move kprobe selftest function to separate compile unit 2018-07-30 18:41:04 -04:00
trace_kprobe_selftest.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00
trace_mmiotrace.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
trace_nop.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
trace_output.c tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
trace_output.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00
trace_preemptirq.c tracing: More reverting of "tracing: Centralize preemptirq tracepoints and unify their usage" 2018-08-10 15:12:00 -04:00
trace_printk.c tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
trace_probe.c tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
trace_probe.h tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
trace_sched_switch.c tracing: Fix sched switch start/stop refcount racy updates 2020-02-11 04:33:55 -08:00
trace_sched_wakeup.c kernel/trace: Fix do not unregister tracepoints when register sched_migrate_task fail 2020-01-14 20:06:59 +01:00
trace_selftest.c Tracing updates for 4.15: 2017-11-17 14:58:01 -08:00
trace_selftest_dynamic.c ftrace: Mark function tracer test functions noinline/noclone 2018-01-23 15:57:29 -05:00
trace_seq.c tracing: Add SPDX License format tags to tracing files 2018-08-16 19:08:06 -04:00
trace_stack.c tracing: Have stack tracer compile when MCOUNT_INSN_SIZE is not defined 2020-01-14 20:06:59 +01:00
trace_stat.c tracing: Fix very unlikely race of registering two stat tracers 2020-02-24 08:34:39 +01:00
trace_stat.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00
trace_syscalls.c Tracing updates for 4.15: 2017-11-17 14:58:01 -08:00
trace_uprobe.c tracing/uprobes: Fix output for multiple string arguments 2019-02-20 10:25:48 +01:00
tracing_map.c tracing: Have the histogram compare functions convert to u64 first 2020-01-09 10:19:02 +01:00
tracing_map.h tracing: Fix SPDX format headers to use C++ style comments 2018-08-16 19:08:06 -04:00