From 926bdb3bb5c241da141931a2b4741d0fe017b924 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Desbiens?= Date: Thu, 17 Sep 2026 22:31:43 -0400 Subject: [PATCH] Stopped the SMP Linux port losing a mutex wake-up to a suspend signal The SMP Linux port suspends a thread with a signal whose handler calls sigsuspend and does not return until the thread is resumed. That signal can arrive while the thread is parked in pthread_mutex_lock on _tx_linux_mutex; the port knows it can, because _tx_linux_mutex_obtain sets tx_thread_linux_mutex_access around the lock call for exactly this case, and nothing anywhere reads that flag. glibc waits for a contended mutex in a loop that re-arms the futex wait after a signal, and this handler never returns to it. The next release hands its wake-up to that thread, which will not act on it, and any other thread parked on the mutex is never woken, leaving the mutex free with waiters on it. That deadlocks the process: the only thread that can resume the suspended one is the scheduler, and the scheduler takes this mutex on every pass. _tx_linux_mutex_obtain now waits with pthread_mutex_timedlock and retries, so the wait is re-armed every TX_LINUX_MUTEX_RETRY_NSEC and a lost wake-up costs one retry period instead of the process. The period is one millisecond, half the scheduler's own idle period. Nothing else changes. Four hung processes captured untraced, across three tests, show the same state: a thread in sigsuspend on top of pthread_mutex_lock, the scheduler blocked in pthread_mutex_lock, and the mutex reading free. Standalone, threadx_smp_random_resume_suspend_exclusion_test hung 3 times in 100 runs and 2 in 55 before the change and 0 in 400 after it. Assisted-by: Claude Code (Opus 5) --- ports_smp/linux/gnu/inc/tx_port.h | 10 +++++ ports_smp/linux/gnu/src/tx_thread_schedule.c | 41 +++++++++++++++++++- 2 files changed, 49 insertions(+), 2 deletions(-) diff --git a/ports_smp/linux/gnu/inc/tx_port.h b/ports_smp/linux/gnu/inc/tx_port.h index dc1aeade2..53a4dc03f 100644 --- a/ports_smp/linux/gnu/inc/tx_port.h +++ b/ports_smp/linux/gnu/inc/tx_port.h @@ -513,6 +513,16 @@ typedef struct } TX_LINUX_MUTEX; +/* Define how long a thread waits on the Linux mutex before retrying. A thread + parked on the mutex can be suspended by the port's signal handler and so never + act on the wake-up the next release sends it, which leaves the wake-up lost + and every other waiter parked on a mutex that is free. */ + +#ifndef TX_LINUX_MUTEX_RETRY_NSEC +#define TX_LINUX_MUTEX_RETRY_NSEC 1000000 +#endif + + /* Define Linux-specific critical section APIs. */ void _tx_linux_mutex_obtain(TX_LINUX_MUTEX *mutex); diff --git a/ports_smp/linux/gnu/src/tx_thread_schedule.c b/ports_smp/linux/gnu/src/tx_thread_schedule.c index 0c3e8e326..0b6eba259 100644 --- a/ports_smp/linux/gnu/src/tx_thread_schedule.c +++ b/ports_smp/linux/gnu/src/tx_thread_schedule.c @@ -292,6 +292,8 @@ void _tx_linux_mutex_obtain(TX_LINUX_MUTEX *mutex) TX_THREAD *thread_ptr; pthread_t current_thread_id; UINT i; +INT linux_status; +struct timespec ts; /* Pickup the current thread ID. */ current_thread_id = pthread_self(); @@ -343,8 +345,43 @@ UINT i; thread_ptr -> tx_thread_linux_mutex_access = TX_TRUE; } - /* Get the Linux mutex. */ - pthread_mutex_lock(&mutex -> tx_linux_mutex); + /* Get the Linux mutex. The wait is timed and retried rather than left + to pthread_mutex_lock, because a thread can be signalled into the + port's suspend handler while it is parked on this mutex. That handler + does not return until the thread is resumed, so the wake-up the next + release sends is delivered to a thread that never retries and is lost. + Any other thread parked on the mutex then waits on a mutex that is + free, which on this port deadlocks the whole process: the thread that + can resume the suspended one is the scheduler, and the scheduler takes + this mutex on every pass. Retrying on a timeout costs nothing when + the mutex is handed over normally, and turns that lost wake-up into a + delay of at most the retry period. */ + do + { + + /* Set the deadline for this attempt. */ + clock_gettime(CLOCK_REALTIME, &ts); + ts.tv_nsec = ts.tv_nsec + TX_LINUX_MUTEX_RETRY_NSEC; + if (ts.tv_nsec >= 1000000000) + { + + ts.tv_nsec = ts.tv_nsec - 1000000000; + ts.tv_sec++; + } + + linux_status = pthread_mutex_timedlock(&mutex -> tx_linux_mutex, &ts); + + /* Anything but the deadline expiring is a real failure to obtain the + mutex, so stop retrying and record it. */ + if ((linux_status != 0) && (linux_status != ETIMEDOUT)) + { + + /* Increment the system error counter. */ + _tx_linux_system_error++; + break; + } + + } while (linux_status != 0); /* At this point we have the mutex. */