GitHub Actions exit code 137: prove OOM before adding RAM

Collect evidence in four passes

1. Check whether GitHub canceled the job

Open the workflow run summary before reading the last process line in isolation. A canceled run, a superseded concurrency group, a manual cancellation, or a duration that reaches timeout-minutes points to job control rather than memory.

# Inspect run and job conclusions from a trusted local shell
gh run view RUN_ID \
  --json conclusion,status,createdAt,startedAt,updatedAt,jobs

# Download the full log archive for the same run
gh run view RUN_ID --log-failed

GitHub cancellation starts with interrupt and termination signals, then kills the process tree if it does not exit. That final forced kill can resemble an unexplained 137 in a child process.

2. If Docker is involved, inspect container state

docker inspect CONTAINER_ID \
  --format 'exit={{.State.ExitCode}} oom={{.State.OOMKilled}} error={{.State.Error}}'

docker ps -a --filter 'exited=137'

OOMKilled=true is direct container evidence. A container exit of 137 with OOMKilled=false still requires investigation because Docker also reports 137 after other SIGKILL paths.

3. On a self-hosted runner, read cgroup and kernel evidence

# cgroup v2 counters
cat /sys/fs/cgroup/memory.events 2>/dev/null || true

# Host-level OOM messages may require elevated log permissions
dmesg -T 2>/dev/null | grep -Ei 'out of memory|oom-kill|killed process' || true

# Current memory and the largest resident processes
free -h
ps -eo pid,ppid,rss,comm,args --sort=-rss | head -20

In cgroup v2, a rising oom_kill counter records processes killed by an OOM killer in that cgroup. A max or oom event without oom_kill is memory pressure, but not proof that this process was killed.

4. Measure the failing command, not the whole job average

/usr/bin/time -v your-build-command

# Useful fields:
# Maximum resident set size
# Elapsed wall clock time
# Exit status

A job can look healthy before one compiler, linker, test shard, or bundle step spikes. Measure the smallest reproducible command and compare the same revision across repeated runs.

Apply the fix that matches the evidence

Reduce peak concurrency before reducing test coverage

# Jest
npx jest --maxWorkers=2

# Make
make -j2

# Go package build/test parallelism
go test -p=2 ./...

This keeps the same work while limiting simultaneous memory peaks. Record the before and after maximum resident set size; otherwise a passing rerun may only be noise.

Split independent workloads into separate jobs

Compiling, testing, linting, and packaging in one job can retain caches and worker processes at the same time. Separate jobs can give each phase a clean runner, but artifact transfer and extra minutes have a cost. Split only where the phase boundary is real and outputs can be verified.

Tune a runtime limit only with headroom

# Example for a Node process after measuring the runner budget
NODE_OPTIONS=--max-old-space-size=4096 npm run build

A larger JavaScript heap can make the total process exceed the runner sooner. Leave memory for native allocations, child processes, the operating system, and service containers. Do not copy a heap number without checking the current runner specification and the job's other processes.

Use a larger or self-hosted runner last

GitHub publishes current standard and larger-runner specifications, and the available sizes vary by repository, plan, operating system, and architecture. A larger runner is justified when the workload is already bounded, measured, and still exceeds the supported machine. It is not a substitute for an unbounded worker pool or a leaked process.

Add a bounded memory trace to the failing step

- name: Run build with memory evidence
  shell: bash
  run: |
    set -euo pipefail

    monitor_memory() {
      while true; do
        date -u '+%Y-%m-%dT%H:%M:%SZ'
        free -h
        ps -eo pid,ppid,rss,comm --sort=-rss | head -15
        sleep 10
      done
    }

    monitor_memory &
    monitor_pid=$!
    trap 'kill "$monitor_pid" 2>/dev/null || true' EXIT

    /usr/bin/time -v npm run build

Keep this diagnostic step temporary or route its output to a short-retention artifact. Process lists and verbose logs can reveal repository paths or command arguments, so review the output before sharing it publicly. Never print secrets or full environments.

Common questions

Is exit code 137 always an out-of-memory failure?

No. It indicates SIGKILL termination. OOM is one possible sender, but Docker, GitHub runner cancellation, a timeout, or another supervisor can also kill the process.

Why is there no JavaScript stack trace?

SIGKILL cannot be caught or handled by the target process. The process has no opportunity to flush a stack trace, which is why runner, container, cgroup, and peak-memory evidence matters.

Should I immediately increase Node max-old-space-size?

Not without measurement. The V8 heap is only part of total memory. Native modules, bundlers, child processes, and the OS also consume RAM, so a larger heap can move the failure rather than remove it.

When should I pay for a larger runner?

After confirming a repeatable memory peak, bounding concurrency, removing retained processes, and checking the current runner specification. Keep the memory trace as a regression guard after resizing.