[CI]: `mypy-primer(0)` timeout due to long checking of `colour`

[randolf-scholz]

In several recent PRs the mypy-primer (0) can take close to, or over an hour causing timeout due to long checking on the colour package.

• Try to fix problems caused by in-place modification of typevar defaults #19382: 55 min, over 50 min spent on colour
• Fix Enum.value inference for Enums with @cached methods #19374: 55 min, over 50 min spent on colour
• perf: add __slots__ to SubtypeContext #19397: 58 min, over 56 min spent on colour
• Draft: Solve intermediate variable bug #19399: timeout, over 56 min spent on colour

[sterliakov]

Primer can be rebalanced, there are weights assigned to every project proportional to the check time. colour might need its own shard.

[randolf-scholz]

But these have to be defined over in https://github.com/hauntsaninja/mypy_primer, right? So I should move this issue there?

[randolf-scholz]

I tried running mypy locally on a fresh clone of colour, and it doesn't take nearly as long. It does take >600MB, so maybe the runner goes into swap if it runs mypy on multiple repos in parallel?

Command exited with non-zero status 1
	Command being timed: "mypy colour/ --no-incremental"
	User time (seconds): 83.03
	System time (seconds): 0.45
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 1:23.51
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 622748
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 201966
	Voluntary context switches: 11
	Involuntary context switches: 908
	Swaps: 0
	File system inputs: 51825
	File system outputs: 199294
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 1

[sterliakov]

How beefy is your PC? I tried running mypy on colour before, and it did take 30+ mins.

Yes, the weights should be adjusted in that repo, you can try opening a PR there or pinging @hauntsaninja for comments:) AFAIC I brought this up before and we agreed that rebalancing is the way to go, optionally along with adding a 6th shard for colour alone.

[randolf-scholz]

I do have a relatively recent build with an intel 265K and DDR5. Though I did run with mypy 1.16.1 rather than the most recent master, so maybe there is a regression.

[sterliakov]

Hm. Did you run compiled or interpreted mypy? A month or so ago I got ~50 mins using interpreted mypy on a laptop with Intel Core i3 with 32G DDR4 RAM.

If I remember correctly, primer runs interpreted mypy too.

[randolf-scholz]

So I did a rerun (using --no-incremental and deleting .mypy_cache)

• With mypy 1.18.0+dev.4a427e9f1ac8a007d5c9bea946cfeb9e163cfa89 (compiled: no) it takes roughly 8 min 30 sec
• With mypy 1.18.0+dev.4a427e9f1ac8a007d5c9bea946cfeb9e163cfa89 (compiled: yes) roughly 1 min 20 sec

The former makes the 50+ min runtime on a weak machine seem more plausible.

[JelleZijlstra]

Out of curiosity, do we have any idea why mypy is so slow and/or memory hungry on this project? I took a quick look and it looks like it's big but not extremely huge.

[randolf-scholz]

It produces a large number of errors when type checked, and makes use of numpy/scipy which uses complex protocol types, so I think that contributes to it.

mypy.log

[randolf-scholz]

And it uses large unions of literals:

colour/models/rgb/tests/test_rgb_colourspace.py:365: error: Argument 2 to "XYZ_to_RGB" has incompatible type "ndarray[tuple[int, ...], dtype[Any]]"; expected "RGB_Colourspace | Literal['ACES2065-1', 'ACEScc', 'ACEScct', 'ACEScg', 'ACESproxy', 'ARRI Wide Gamut 3', 'ARRI Wide Gamut 4', 'Adobe RGB (1998)', 'Adobe Wide Gamut RGB', 'Apple RGB', 'Best RGB', 'Beta RGB', 'Blackmagic Wide Gamut', 'CIE RGB', 'Cinema Gamut', 'ColorMatch RGB', 'DCDM XYZ', 'DCI-P3', 'DCI-P3-P', 'DJI D-Gamut', 'DRAGONcolor', 'DRAGONcolor2', 'DaVinci Wide Gamut', 'Display P3', 'Don RGB 4', 'EBU Tech. 3213-E', 'ECI RGB v2', 'ERIMM RGB', 'Ekta Space PS 5', 'F-Gamut', 'FilmLight E-Gamut', 'ITU-R BT.2020', 'ITU-R BT.470 - 525', 'ITU-R BT.470 - 625', 'ITU-R BT.709', 'ITU-T H.273 - 22 Unspecified', 'ITU-T H.273 - Generic Film', 'Max RGB', 'N-Gamut', 'NTSC (1953)', 'NTSC (1987)', 'P3-D65', 'PLASA ANSI E1.54', 'Pal/Secam', 'ProPhoto RGB', 'Protune Native', 'REDWideGamutRGB', 'REDcolor', 'REDcolor2', 'REDcolor3', 'REDcolor4', 'RIMM RGB', 'ROMM RGB', 'Russell RGB', 'S-Gamut', 'S-Gamut3', 'S-Gamut3.Cine', 'SMPTE 240M', 'SMPTE C', 'Sharp RGB', 'V-Gamut', 'Venice S-Gamut3', 'Venice S-Gamut3.Cine', 'Xtreme RGB', 'aces', 'adobe1998', 'prophoto', 'sRGB'] | str"  [arg-type]

[sterliakov]

It's extremely overload-heavy due to numpy usage, checking nested overloads and numpy arithmetic is very slow by itself (see #14978 for a pathological case - 5-10 numpy operands can bring mypy to halt).