Altering callmaps
Intro
One of the first things most contributors start with is proposing changes to callmaps.
Callmap is a data file (formatted as a PHP file returning an array) that tells
Psalm what arguments function/method takes and what it returns. Psalm loads
dictionaries/CallMap_<PHP major version><PHP minor version>.php for the analyzed
PHP version (CallMap_84.php for PHP 8.4).
These files are generated: never edit them by hand. bin/stubs/gen_callmap.php builds
them from two sources:
dictionaries/autogen/CallMap_*.php: the signatures read by reflection from PHP and a set of extensions, one file per PHP version;dictionaries/override/: hand written corrections.CallMap.phpholds the signatures of the newest PHP version, and the delta filesCallMap_*_delta.phphold how they changed in each version.
An override entry wins over the reflected one. Overrides equal to the reflected signature in every version are dropped.
Changing a signature
- Edit the entry in
dictionaries/override/CallMap.php(to add a missing one, copy it fromdictionaries/autogen/CallMap_$latest.php). - Run
php -d memory_limit=-1 bin/stubs/gen_callmap.php. It regeneratesdictionaries/CallMap_*.phpand rewrites the delta files.
The edit applies to every older version up to the first delta that changes the
function. Narrowing the parameter of array_key_first this way changes
CallMap_80.php to CallMap_86.php and the new side of its changed entry in
CallMap_80_delta.php, while CallMap_74.php keeps the old side. To change an
older signature, edit the old side of the entry in that delta.
Regenerating the reflected callmaps
bin/stubs/gen_callmap.sh (composer gen_callmap, part of composer build) runs
bin/stubs/gen_base_callmap.php inside one Docker image per PHP version
(ghcr.io/<owner>/psalm:internal_stubs_<version>), then gen_callmap.php.
bin/stubs/build_docker.php builds those images and generates their Dockerfile on
the fly.
To add the types of an extension, add it to EXTENSIONS in bin/stubs/build_docker.php
(or to INCLUDES, for some PHP versions only) with its version source in SOURCES.
PIE_PACKAGES lists the extensions installed with PIE, and EXCLUDES the ones that
don't build on a PHP version yet.
Every push runs the "Build docker image for stubs" workflow: it builds the images,
runs composer build, and when a committed file is out of date, commits the difference
to the pushed branch and uploads it as the composer-build-patch artifact. Pull before
pushing again. If the commit can't be pushed, bin/ci/apply-composer-build-patch.sh
applies the artifact to your checkout (set PSALM_REPO=<you>/psalm for a run on your fork).
Checking a callmap against reflection
composer verify-callmap runs tests/Internal/Codebase/InternalCallMapHandlerTest.php, which compares the
callmap with what reflection of the installed PHP and its extensions says.
Functions that don't pass yet are listed in that test ($ignoredFunctions, and for return types only
$ignoredReturnTypeOnlyFunctions). To fix one:
- Remove its entry from the ignore list and run
composer verify-callmap. - Fix the errors it reports in the callmap (and in the deltas, if the signature changed in an older version).
- Run
composer verify-callmapagain, then open the PR.
An entry that is already right must not be on the list: the test fails with Remove 'name' from
InternalCallMapHandlerTest::$ignoredFunctions.
Full callmap format
Each entry keeps its analysis (or "doc") return type at index 0. The optional
index -1 records the native or tentative return type extracted from PHP:
'strlen' => [
0 => 'int<0, max>',
-1 => 'int',
'string' => 'string',
],
'amqpvalue::toamqpvalue' => [
0 => 'mixed',
],
'jsonserializable::jsonserialize' => [
-1 => 'mixed',
-2 => 'tentative',
],
'limititerator::getposition' => [ // PHP 8.0: no native or tentative return signature
0 => 'int',
-2 => 'none',
],
Only the signature return type imposes a native return-type requirement on an
implementation. -1 => 'mixed' is an explicit signature, not an absent type.
Optional index -2 records its provenance:
-1 |
-2 |
Meaning |
|---|---|---|
| present | absent | Native return signature |
| present | tentative |
Tentative return signature, available since PHP 8.1 |
| absent | none |
Reflection confirmed there is no return signature in this PHP version |
| absent | absent | Not reflected by the generator; the stub or runtime may provide a signature |
#[ReturnTypeWillChange] suppresses tentative return-signature checks, including
an explicitly incompatible return type. It does not suppress a native signature.
Stubs and index 0 still provide the types used to analyse calls; PHPDoc
compatibility checks are separate from native signature checks.
Generated files omit index 0 when its type equals index -1; the signature then
also supplies the analysis return type. Read it with $entry[0] ?? $entry[-1];
getCallMap() preserves the compact entries without recreating index 0.
Edit the analysis return type in the overrides, then regenerate. Signature return types come from reflection and are retained independently of overrides and alternate signatures; the generator does not copy analysis refinements into them.
Reflection coverage
Each image must load every extension in the version-specific manifest returned by
php bin/stubs/build_docker.php --extensions-plain <version>. Generation fails
before writing a callmap if its PHP version or a required extension is wrong.
dictionaries/autogen/CallMapExtensions_*.php records the extension of each
reflected API. These manifests describe coverage; they do not list every PHP extension.
For example, the current images do not load bz2, calendar or dba.
Their handwritten entries can therefore lack reflected signature metadata.
The merger preserves overrides for APIs absent from an image. An untyped downloaded PHPStorm stub does not establish that an extension has no native return type. For an API without generated metadata, Psalm first uses its stub signature. Runtime reflection can supply a missing signature only when the analysed PHP major/minor matches the PHP running Psalm; it never imports a newer runtime signature into an older target. Results for these uncovered APIs can consequently depend on the installed extensions and their versions. A different target version needs a stub or generated metadata for that version.
A callmap (dictionaries/CallMap_*.php, and dictionaries/override/CallMap.php) has function/method
names as keys and an array representing the corresponding signature as a value.
Entries with string keys represent function/method parameters. Parameter name for an
optional parameter is postfixed with =, references with &/&r_/&w_/&rw_
(depending on the read/write meaning of the reference param) and
variadic args are prefixed with ....
Callmaps also support function aliases: aliases are very useful to specify that a certain function behaves differently according to the parameter types.
For example, the following aliases are currently in use in the callmap to specify
that the version_compare function can be called with an operator parameter,
in which case it will return a boolean; otherwise it will return an integer.
[
'version_compare' => [
0 => 'bool',
'version1' => 'string',
'version2' => 'string',
'operator' => '\'!=\'|\'<\'|\'<=\'|\'<>\'|\'=\'|\'==\'|\'>\'|\'>=\'|\'eq\'|\'ge\'|\'gt\'|\'le\'|\'lt\'|\'ne\'|null',
],
'version_compare\'1' => [
0 => 'int',
'version1' => 'string',
'version2' => 'string',
],
]
Note: the above example doesn't provide almost any useful information for type inference;
in fact, the real logic for return type inference is contained in the VersionCompareReturnTypeProvider.
The callmap is mainly useful when treating functions and methods as callable values, for example:
<?php
function naive_version_compare(string $a, string $b, ?string $operator = null): int|bool {
return 0;
}
$a = ["1.0", "2.0"];
// OK
usort($a, "version_compare");
// InvalidArgument: Argument 2 of usort expects callable(string, string):int, but impure-callable(string, string, null|string=):(bool|int) provided
usort($a, "naive_version_compare");
The first usort succeeds, because psalm chooses the correct alias to use between the two provided in the callmap.
The second usort fails (equivalent to the non-split return type of version_compare inferred by reflection), because the return type is a union of the two possible signatures of version_compare.
When you have multifaceted functions like these, it's a very good idea to at least define a templated stub in stubs/ for them, or a custom return type provider for even more complex logic, not representable with templates/conditional types/etc in a stub.
Also note that bin/stubs/gen_callmap.php has some validation logic which will re-add back removed parameters in overridden aliased callmaps: to avoid this, explicitly whitelist aliased functions by editing assertParameter in bin/stubs/gen_callmap_utils.php, and eventually bin/stubs/gen_callmap.php as needed.
Delta file format
Delta files (named CallMap_<PHP major version><PHP minor version>_delta.php)
list changes that happened in the corresponding PHP version. There are
three section with self-explanatory names: added (for functions/methods that
were added in that PHP version), removed (for those that were removed) and
changed.
Entry format for removed and added section matches that of a full callmap,
while changed entries list old and new signatures.
How the generator uses delta files
bin/stubs/gen_callmap.php starts from dictionaries/override/CallMap.php (the
newest PHP version) and applies each delta in reverse to get the overrides of the
version before it: entries in added are removed, those in removed are added
back, and changed.new signatures are replaced with changed.old. It merges the
overrides of each version over the reflected callmap of that version, then rewrites
the delta files from the result, so they always match override/CallMap.php.
The "Build docker image for stubs" workflow fails when a committed callmap differs from what the generator produces.
Adding a function
The functions and methods of PHP and of the extensions in the stub images come from
reflection, so a new one reaches the callmaps without an override: array_is_list()
(PHP 8.1) has no override entry. Add one only to refine its types, as described in
Changing a signature.