Magento 2 Memory Leaks in Cron Jobs & Queue Consumers: Find, Fix and Prevent Them

Cron jobs and queue consumers are the background engine of a Magento 2 store — and the most common place where memory problems hide. A web request lives for a second and dies, taking its garbage with it. A queue consumer or a heavy cron job can run for hours, and every leak it accumulates stays in the process until it crashes.

The typical story: the consumer worked fine on Monday. By Wednesday it has processed 40,000 messages, its RSS has climbed from 120 MB to 900 MB, and at 2 AM it slams into memory_limit and dies. Messages pile up, the email backlog grows, and the only clue in the logs is Allowed memory size of 262144000 bytes exhausted with a stack trace that points nowhere useful.

This guide explains why Magento leaks in long-running processes, how to prove it, and a fix playbook you can apply today.

Why PHP processes grow without bound

PHP is not inherently leak-prone for short requests. It frees memory by reference counting: when an object's refcount drops to zero, the memory is reclaimed immediately.

Three things break that model in long-running processes:

1. Circular references. Object A holds a reference to B, B holds one back to A. Neither ever reaches zero. PHP's cycle collector (gc_enable()) exists to break these, but it only helps if it actually runs, and Magento is full of cycles: product ↔ stock item, order ↔ order items, plugin proxies capturing their targets.

2. Static and singleton state. A web request dies and everything is freed. A consumer doesn't. Anything stored in a static property, a singleton, or the object manager lives until the process restarts. Magento 2 is built on the object manager and singletons, so a single line of extension code that stashes a collection "for later" becomes a permanent retention point.

3. Collection internals. Magento\Framework\Model\ResourceModel\Db\Collection\AbstractCollection keeps every loaded entity in both _items and _itemsById. If your loop loads 100,000 products into one collection, all 100,000 objects stay referenced until the collection is cleared or falls out of scope — which in a long-lived loop may be never.

The Magento-specific retention points

• ObjectManager & singleton instances — the DI container keeps whatever it created, for the lifetime of the process.
• Event dispatch — observers that capture objects into static arrays or into other long-lived objects.
• Loggers and the profiler — see below, this one is easy to hit accidentally.
• Message queue internals — consumer loops that keep the last message object (or an array of processed messages) referenced until the next iteration.
• Caches with static backing — any "cache in a static array" pattern (common in third-party modules and cache warmers) grows one entry at a time, forever.

Detect it: measure RSS over time, not peak

A leak is a trend, so a single measurement proves nothing. You need a time series:

# Watch a running consumer's memory grow (or not)
CONSUMER_PID=$(pgrep -f "queue:consumers:start" | head -1)
while true; do
  ps -o pid,rss,vsz,etime,cmd -p "$CONSUMER_PID"
  sleep 10
done

For cron runs, add a memory checkpoint at the end of your own jobs:

$this->logger->info(sprintf(
    'Job %s done: peak=%dMB, end=%dMB',
    $jobName,
    memory_get_peak_usage(true) / 1048576,
    memory_get_usage(true) / 1048576
));

Getting the trend lets you classify the shape:

• Stable plateau, then a step up — an event-based retention (e.g. every N messages something is cached).
• Linear climb — a per-iteration leak; each message or row retains a fixed amount.
• Sawtooth but drifting upward — the cycle collector runs and reclaims, but retention outpaces it.
• Peak-only tools lie — /usr/bin/time -v shows peak RSS for the whole run, which for a well-behaved job is high anyway. For leak hunting, profile over time with Blackfire or xhprof memory graphs, or sample RSS as above.

The classic leak scenarios on real stores

1. Consumers running forever. bin/magento queue:consumers:start <name> without --max-messages runs until killed. Every per-message retention compounds hour after hour. This is the single most common cause of consumer OOMs. Restart them on a schedule (see the playbook).

2. The profiler left on. If dev/debug/profiler is enabled (or a profiler module is active), Magento\Framework\Profiler accumulates timing data for every event, timer, and query in memory. In a long-running process this alone can add hundreds of MB. It's a dev tool — keep it off in production, and never enable it on a consumer that you don't plan to restart.

3. Whole-collection loops. Loading a full collection and iterating it is fine in a web request, lethal in a cron job:

// Bad: every product stays referenced in the collection
$products = $this->productCollectionFactory->create();
$products->addAttributeToSelect('*'); // full EAV rows
foreach ($products as $product) {
    $this->doSomething($product);
}

4. Import scripts that slurp files. file($csvPath) on a 500 MB export and $product->load($sku) per row without clearing — peak memory explodes and nothing gets reclaimed between rows.

5. Indexer or scheduler plugins. Reindex processes already batch internally in 2.4.x, but extensions hooked onto indexer events (catalog permissions, staging, custom price dimensions) often retain per-batch data in static arrays. Profile the reindex with xhprof and look at who holds references after each batch.

The fix playbook

1. Bound every long-running process

For consumers, never rely on "it's been fine for months":

bin/magento queue:consumers:start order_emails --max-messages=500

--max-messages exists since Magento 2.3 and cleanly exits after N messages; recent 2.4 releases also support --max-execution-time. Pair this with a process supervisor (Supervisor, systemd, or the cron_consumers_runner config) that respawns the consumer immediately — then a bounded restart is invisible to your queue. This turns "memory climbs forever" into "memory climbs for 500 messages, then a fresh process starts".

2. Batch and clear collections

$collection = $this->productCollectionFactory->create();
$collection->addAttributeToSelect('*');
$collection->setPageSize(500);
$lastPage = $collection->getLastPageNumber();

for ($page = 1; $page <= $lastPage; $page++) {
    $collection->setCurPage($page);
    foreach ($collection as $product) {
        $this->doSomething($product);
    }
    $collection->clear(); // release _items and _itemsById
}

->clear() is the key line: it drops the internal item arrays so the next batch starts from a clean slate.

3. Unset and collect cycles in custom loops

$processed = 0;
while ($message = $queue->receive()) {
    $this->process($message);
    unset($message);
    if (++$processed % 100 === 0) {
        gc_collect_cycles(); // break circular references now
    }
}

Calling gc_collect_cycles() periodically is cheap insurance in any long-running loop; it lets the cycle collector sweep instead of letting garbage accumulate until the process dies.

4. Ban static retention in custom code

Review your own modules and the offenders from a third-party extension performance audit: replace "static array as cache" with a real cache backend, and never store loaded entities in static properties across iterations. If you genuinely need to reference an object without keeping it alive, PHP 7.4+ WeakReference is the correct tool — but the correct answer is usually "don't retain it at all".

5. Prefer more, shorter workers over one immortal worker

A consumer that runs 24/7 with 2 GB of retained memory is a crash waiting to happen. Five consumers with --max-messages=200 under Supervisor process the same volume, spread risk, and absorb a crash without a backlog. The same logic applies to cron: split heavy jobs into smaller scheduled runs instead of one monolithic window (see the cron optimization guide).

6. Monitor memory as a first-class metric

Sample RSS per process with Prometheus + node_exporter (or your APM), and alert on trend over 1–2 hours, not on absolute thresholds. If a consumer's RSS climbs steadily, that's a leak signal long before it OOMs. Keep an eye on the number of consumers vs memory_limit on the box: with several consumers at 1 GB each, a 4 GB server dies from memory pressure even though no single process crashed.

7. Don't just raise memory_limit

Raising memory_limit converts a 30-minute crash into a 3-hour one, and pushes the problem into the kernel: under sustained pressure the OOM killer may take down MySQL or Redis along with your consumers. Fix the leak or bound the process — the limit is a safety net, not a solution.

Verify the fix

Run the consumer on a baseline, sample RSS every few minutes, and compare before/after:

• Before: RSS climbs ~25 MB per 1,000 messages and never comes back down.
• After: RSS plateaus or sawtooths within a ±50 MB band over 20,000 messages.

If the trend stays flat over a full day and a bounded restart cycle, the leak is contained. Log memory_get_peak_usage(true) at the end of every cron job so a regression shows up in your logging best practices trail before it shows up in an incident.

Quick checklist

• Every queue consumer started with --max-messages (or --max-execution-time) and under a respawning supervisor.
• Cron jobs batch-load collections and call ->clear() per batch.
• No static arrays holding entities across loop iterations; no file() on huge CSVs; streaming imports instead.
• Profiler disabled in production.
• Custom long loops call gc_collect_cycles() periodically.
• RSS trend monitored and alerted, not just peak memory.
• memory_limit treated as a safety net, not a fix.

Memory leaks in cron and queue consumers are an operational problem, not a mystery: measure the trend, bound the process, batch the work, and restart on a schedule. Do that, and the 2 AM OOM becomes a thing of the past — and your message queues and indexers keep running while you sleep.