Skip to content

Fix GH-25379: Preserve array_map() semantics in foreach optimization - #25546

Open
prateekbhujel wants to merge 6 commits into
php:PHP-8.6from
prateekbhujel:fix/gh25379-array-map-foreach-optimization
Open

prateekbhujel wants to merge 6 commits into
php:PHP-8.6from
prateekbhujel:fix/gh25379-array-map-foreach-optimization

Conversation

@prateekbhujel

@prateekbhujel prateekbhujel commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the compiler and runtime regressions reported in GH-25379 inside zend_compile_func_array_map():

  1. Statically known by-value callbacks only (zend_array_map_callback_is_optimizable()):
    Because FE_FETCH_R dereferences elements while array_map() preserves references passed to callbacks, only inline calls whose target function or method is statically known (without walking unlinked parents, since traits or later declarations in the active class can override them) and takes the mapped argument by value. Unresolved, dynamic, abstract, ZEND_ACC2_FORBID_DYN_CALLS, invalid non-static, or by-reference callbacks fall back cleanly to native array_map() before emitting any opcodes.

  2. Clean bailout and evaluation order:
    Validate the callback and PFA arguments before emitting any bytecode, and defer allocating the loop temporary until after checking whether the compiled array argument is an IS_CONST non-array so bailouts never leak temporaries or evaluate subexpressions twice.

Closes GH-25379.

@devnexen

Copy link
Copy Markdown
Member

By-reference callbacks

Not entirely sure about this point, can this be tried ?

--TEST--
GH-25379: array_map() foreach optimization with by-ref callee unknown at compile time
--FILE--
<?php

function test(string $name, Closure $c) {
    echo $name, ': ';
    try {
        echo json_encode($c()), PHP_EOL;
    } catch (Throwable $e) {
        echo $e::class, ': ', $e->getMessage(), PHP_EOL;
    }
}

class P {
    public static function m(&$x) {
        return ++$x;
    }
}

class C extends P {
    public static function viaStatic() {
        return array_map(static::m(...), [1]);
    }
    public static function viaParent() {
        return array_map(parent::m(...), [1]);
    }
    public static function viaSelf() {
        return array_map(self::m(...), [1]);
    }
}

function rec(&$x) {
    static $depth = 0;
    if ($depth++) {
        return ++$x;
    }
    return array_map(rec(...), [1]);
}

test('declared later', fn() => array_map(later(...), [1]));
test('dynamic name', function () {
    $f = 'sort';
    return array_map($f(...), [[3, 1]]);
});
test('static', fn() => C::viaStatic());
test('parent', fn() => C::viaParent());
test('inherited self', fn() => C::viaSelf());
test('recursive', function () {
    $v = 1;
    return rec($v);
});

function later(&$x) {
    return ++$x;
}

?>
--EXPECTF--
declared later:
Warning: later(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[2]
dynamic name:
Warning: sort(): Argument #1 ($array) must be passed by reference, v
[true]
static:
Warning: P::m(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[2]
parent:
Warning: P::m(): Argument #1 ($x) must be passed by reference, value
[2]
inherited self:
Warning: P::m(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[2]
recursive:
Warning: rec(): Argument #1 ($x) must be passed by reference, value
[2]

@prateekbhujel

prateekbhujel commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

@devnexen Good catch. All 6 cases were still throwing Error with the compile-time-only check.

I updated the PR in a204fc4 to handle this in the emitted send opcode instead:

  • Rewrite the loop placeholder's ZEND_SEND_VAL_EX into ZEND_SEND_USER in zend_compile_func_array_map() when the callee's send mode isn't statically known by-value, so by-ref callees warn and wrap like regular array_map() (known by-value callees keep ZEND_SEND_VAL).
  • Extend ZEND_SEND_USER op2 to CONST|UNUSED|NUM (and update compact_literals.c cache-slot accounting) so named placeholders also work for late/dynamic callees, and let optimize_func_calls.c promote ZEND_SEND_USER back to ZEND_SEND_VAL when OPcache resolves a positional non-ref parameter.

All 6 of your cases plus a late named by-ref placeholder are now covered in Zend/tests/array_map_foreach_optimization_gh25379.phpt and pass under ASan + Debug + ZTS with opcache off, on, and JIT enabled.

@prateekbhujel
prateekbhujel force-pushed the fix/gh25379-array-map-foreach-optimization branch from 146a683 to a204fc4 Compare November 18, 2026 08:01
@devnexen

Copy link
Copy Markdown
Member

There seems to be still some issues, "semantic" ones. here a test

--TEST--
GH-25379: array_map() foreach optimization with by-ref callback and reference elements
--FILE--
<?php

function known(&$x) {
    return ++$x;
}

function test(string $name, Closure $c) {
    echo $name, ': ';
    try {
        echo json_encode($c()), PHP_EOL;
    } catch (Throwable $e) {
        echo $e::class, ': ', $e->getMessage(), PHP_EOL;
    }
}

test('known callee', function () {
    $a = [1, 2];
    $r = &$a[0];
    $res = array_map(known(...), $a);
    return [$res, $a];
});

test('callee declared later', function () {
    $a = [1, 2];
    $r = &$a[0];
    $res = array_map(later(...), $a);
    return [$res, $a];
});

function later(&$x) {
    return ++$x;
}

?>
--EXPECTF--
known callee:
Warning: known(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[[2,3],[2,2]]
callee declared later:
Warning: later(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[[2,3],[2,2]]

@prateekbhujel

prateekbhujel commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor Author

Good catch on the reference semantics, @devnexen.

Preserving native array_map() reference behavior inside an inlined FE_FETCH_R loop gets problematic because FE_FETCH_R dereferences the array elements before they reach the callback.

Rather than trying to work around that at runtime with opcode tweaks, I reverted the VM/optimizer changes and added zend_array_map_callback_takes_value() in zend_compile.c. We now only inline when the callback's mapped parameter is statically known to accept by value. If the parameter is by-reference (or the callee cannot be resolved at compile time), we bail out and fall back to the native array_map() call.

Your test cases, along with tests for static methods and functions declared later in the file, are added in Zend/tests/array_map_foreach_optimization_gh25379_reference_elements.phpt and passing.

Mind taking another look when you get a chance?

@devnexen

Copy link
Copy Markdown
Member

I do think you re getting there but will have a better look this week end

@devnexen

Copy link
Copy Markdown
Member

what about this one ?

--TEST--
GH-25379: array_map() foreach optimization with by-ref parameter added by the child method
--FILE--
<?php

class P {
    public static function m($x) {
        return $x + 1;
    }
}

trait T {
    public static function m($y, &...$x) {
        return ++$x[0];
    }
}

class DeclaredLater extends P {
    public static function positional() {
        $a = [1, 2];
        $r = &$a[0];
        $res = array_map(self::m(0, ?), $a);
        return [$res, $a];
    }
    public static function named() {
        $a = [1, 2];
        $r = &$a[0];
        $res = array_map(self::m(0, x: ?), $a);
        return [$res, $a];
    }
    public static function m($y, &$x = null) {
        return ++$x;
    }
}

class FromTrait extends P {
    use T;

    public static function variadic() {
        $a = [1, 2];
        $r = &$a[0];
        $res = array_map(self::m(0, ?), $a);
        return [$res, $a];
    }
}

function test(string $name, Closure $c) {
    echo $name, ':', PHP_EOL;
    try {
        echo json_encode($c()), PHP_EOL;
    } catch (Throwable $e) {
        echo $e::class, ': ', $e->getMessage(), PHP_EOL;
    }
}

test('declared later, positional', DeclaredLater::positional(...));
test('declared later, named', DeclaredLater::named(...));
test('from trait, variadic', FromTrait::variadic(...));

?>
--EXPECTF--
declared later, positional:

Warning: DeclaredLater::{closure:pfa:DeclaredLater::positional():%d}(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[[2,3],[2,2]]
declared later, named:

Warning: DeclaredLater::{closure:pfa:DeclaredLater::named():%d}(): Argument #1 ($x) must be passed by reference, value given in %s on line %d
[[2,3],[2,2]]
from trait, variadic:

Warning: FromTrait::{closure:pfa:FromTrait::variadic():%d}(): Argument #1 ($x0) must be passed by reference, value given in %s on line %d
[[2,3],[2,2]]

@prateekbhujel

Copy link
Copy Markdown
Contributor Author

Another great catch, @devnexen.

Walking up parent on unlinked classes was the root cause — for CG(active_class_entry) or a class using traits, ce->function_table doesn't have methods declared later or imported from traits yet, so climbing to P resolved P::m() instead of the overriding by-ref method.

In 1bf9f93, I removed the parent walk so we only look up directly on ce (matching zend_compile_static_call() — linked classes already have inherited and trait methods in function_table, while unlinked ones only have their own already-compiled methods). While auditing the helper, I also fixed a couple of related edge cases (parent:: inside a free function where CG(active_class_entry) is NULL, pre-bound literals targeting by-ref parameters, and non-static methods called statically on empty arrays) and dropped the now-redundant callee node plumbing.

Your cases and the extra edge cases are in Zend/tests/array_map_foreach_optimization_gh25379_reference_elements.phpt, and the array_map, PFA, and opcache test suites pass under ASan + Debug + ZTS.

@devnexen

Copy link
Copy Markdown
Member

definitely looks better however I want @TimWolla to have a look at it too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants