Building a Parallel C++ Source Parser: jthread, stop_token, and the Deadlock I Didn't See Coming

I've been building RAAG, a platform that analyzes architectural health across large codebases. It parses a repository, builds a dependency graph, computes coupling metrics, and uses that graph to scope AI-assisted refactoring to only the files a change can actually reach.

This post is about the first layer: the C++20 extraction engine that turns a directory of source files into structured data, fast.

Here's what it does on 579 files:

3.69x speedup on 8 cores. 1,099,446 AST nodes extracted, zero failures.

The interesting part isn't the number. It's the three things that nearly stopped it working at all.

Why C++ for this layer

The analytics live in Python — graph algorithms and orchestration benefit from library maturity more than from raw speed. Extraction is different. It's I/O and CPU bound, it runs over every file in the repository, and it needs no ecosystem breadth.

Running an interpreted parser over a million-line codebase produces wall-clock times nobody tolerates in a pre-commit hook.

So the split is deliberate:

Source files
     │
     ▼
┌──────────────────┐
│  SAMPLE ENGINE   │   C++20 · Tree-sitter · std::jthread
│  (extraction)    │
└────────┬─────────┘
         │  versioned binary snapshot
         ▼
┌──────────────────┐
│   TUNE ENGINE    │   Python · NetworkX
│   (analytics)    │
└────────┬─────────┘
         │  dependency graph + metrics
         ▼
┌──────────────────┐
│  MASTER ENGINE   │   Python · vector store · LLM
│ (orchestration)  │
└──────────────────┘

C++20 for the throughput-bound stage, Python for everything above it, with a versioned binary contract between them.

Parsing is embarrassingly parallel

Files are independent. Nothing about parsing file A depends on file B. That makes this close to the ideal parallel workload — the only shared state is the results container.

The naive approach is one thread per file. That's badly wrong at repository scale: thousands of OS threads, each with a stack of a few hundred kilobytes of virtual memory, and a scheduler forced to context-switch between far more runnable threads than there are cores. Throughput collapses.

A pool fixes this. A fixed number of workers, matched to hardware concurrency, fed from a queue. Thread creation is paid once.

1. std::jthread earns its place

C++20's std::jthread isn't a nicer name for std::thread. It fixes a genuinely hostile default.

std::thread calls std::terminate if it's destroyed while still joinable.

Every path out of a scope holding a thread has to join or detach — including paths taken during exception unwinding, which is exactly where people forget.

jthread joins in its destructor and carries a std::stop_token. My pool's destructor is:

ThreadPool::~ThreadPool() {
    for (std::jthread& worker : workers_) {
        worker.request_stop();
    }
    task_available_.notify_all();
    // No join loop. ~jthread joins.
}

2. The deadlock I didn't see coming

This one cost me real debugging time.

Cancellation has to be cooperative. There's no safe way to kill a thread externally — it might hold a lock, which would stay locked forever, or be halfway through modifying shared state. So the requester sets a flag, and the thread checks it and exits on its own terms. std::stop_token standardizes that flag.

But here's the problem: a worker with nothing to do is blocked on a condition variable.

Destructor:  request_stop()  ──►  ???
                                   │
Worker:      blocked on cv.wait()  │  never wakes
                                   ▼
                              deadlock

If a stop request arrives while a worker is blocked, it has to wake up. Otherwise the destructor waits forever for a thread that's waiting for work that will never arrive.

Plain std::condition_variable has no overload taking a stop_token. Only std::condition_variable_any does.

task_available_.wait(lock, stop, [this] { return !tasks_.empty(); });

if (tasks_.empty()) {
    return;   // Woken by a stop request with nothing left to do.
}

The stop-aware wait registers a stop callback that notifies the variable when stop is requested, and returns false if it woke for that reason rather than because the predicate became true.

Swap in the plain condition_variable and everything compiles. It just hangs on shutdown.

3. Member declaration order is a correctness requirement

Class members are destroyed in reverse declaration order.

class ThreadPool {
    // ...
    std::mutex mutex_;
    std::condition_variable_any task_available_;
    std::queue<std::function<void()>> tasks_;

    // Declared LAST, therefore destroyed FIRST.
    // Workers stop and join while the mutex they wait on is still alive.
    // Moving this line up is a use-after-free.
    std::vector<std::jthread> workers_;
};

The worker vector is declared last, which means it's destroyed first — every jthread stops and joins while the mutex and condition variables they use are still alive.

Move that member above the mutex and you're destroying a mutex while threads are still waiting on it. That's a use-after-free: undefined behaviour that may pass every test you write and crash under production load.

I have a comment on that member saying exactly this, because it looks like arbitrary ordering to anyone reading it fresh.

Exceptions must not escape a worker

A task is arbitrary caller code. If an exception propagates out of a thread's entry function, the standard mandates std::terminate — the whole process dies.

One unparseable file taking down an analysis run over thousands of files is not acceptable behaviour.

try {
    task();
} catch (...) {
}

Swallowing exceptions is usually a smell. Here it's the contract: the pool guarantees a bad task doesn't kill a worker, and recording per-task failures is the caller's responsibility, not the pool's.

Counting tasks, not checking the queue

wait_for_completion() looks like it should wait for the queue to empty.

That's wrong. A task that's been dequeued but is still executing is no longer in the queue, so the wait would return while work was in flight.

The pool tracks an outstanding count, incremented on submit and decremented after execution, and only signals completion at zero.

The pool is only half of thread safety

This is easy to miss when you're pleased with your pool.

The work itself has to tolerate parallelism.

A Tree-sitter parser holds mutable internal state across a parse. Sharing one across threads corrupts trees — and the failure is non-deterministic, showing up as occasional inexplicable parse errors rather than a clean crash. The worst kind of bug.

So each task constructs its own parser. That costs real work the single-threaded path avoids by reusing one parser per language. Giving each worker a thread-local parser instead of constructing one per task would recover it — a known improvement I haven't made yet.

The shared results vector is mutex-guarded, but the lock is only held for the append, never for the parse. Hold it across the expensive work and the mutex serializes everything, and your speedup evaporates.

Serialization: don't memcpy your struct

The parsed ASTs persist to a binary snapshot that the Python layer reads.

The obvious approach — memcpy the node struct to disk — is wrong twice over:

1. The node contains a std::string, so it isn't trivially copyable. Its bytes hold a heap pointer, not the text, and that pointer means nothing on reload.
2. Even for a plain-old-data struct, the byte layout depends on the compiler's padding choices and the host's endianness. A file written by one build might be unreadable by another.

Every field is written explicitly, little-endian, one byte at a time:

const char bytes[4] = {
    static_cast<char>(value & 0xFFu),
    static_cast<char>((value >> 8) & 0xFFu),
    static_cast<char>((value >> 16) & 0xFFu),
    static_cast<char>((value >> 24) & 0xFFu),
};

The file layout:

HEADER
  magic            "RAAG"    4 bytes
  schema_version   uint32
  file_count       uint32

FILE RECORD  × file_count
  path_length      uint32
  path             UTF-8, no terminator
  node_count       uint32

  NODE RECORD  × node_count
    kind               uint8
    name_length        uint32
    name               UTF-8
    byte_start         uint32
    byte_end           uint32
    first_child_index  uint32
    child_count        uint32
    parent_index       int32   (-1 for root)

The magic number catches a wholly wrong file. The schema version catches the subtler case: a file written by an older build whose layout has since changed.

The reader rejects an unknown version rather than reading a different layout as if it were the current one. That path fails silently and produces plausible-looking garbage which surfaces much later as inexplicable graph errors.

Failing loudly at load time is worth far more than succeeding at reading nonsense.

579 files serialize to a 27.6 MB snapshot:

The numbers, and why they're sublinear

Release build, 8 cores, 579 source files from real open-source C++ repositories:

3.69x speedup. Not 8x — and I can tell you exactly why:

Methodology notes

A speedup number is only as good as what it's measured against.

Release build. Debug disables optimization, and unoptimized code flatters parallelism — when serial work is artificially slow, threads look better than they are.

steady_clock, not system_clock. system_clock tracks wall time and can jump — NTP correction, daylight saving — producing nonsensical, possibly negative durations. steady_clock is guaranteed monotonic.

A genuine baseline. The single-threaded path is a real implementation that reuses parsers, not the parallel version limited to one thread. Comparing against a deliberately weakened baseline inflates the number and doesn't survive scrutiny.

What's next

These ASTs become a dependency graph — nodes for files, classes, and functions; edges for imports, calls, and inheritance. From there: afferent coupling, efferent coupling, instability, and cohesion metrics.

That's where the architectural analysis actually starts.

Building RAAG in public. Next post covers turning a million AST nodes into a queryable dependency graph.