Update CI - Drop 3.8 / 3.9

[Erotemic]

Bumping CI versions with xcookie to see if that avoids recent CI errors. This will drop 3.8 support, which is overdue anyway.

I think we may need to re-add the arm entries:

        - python-version: '3.13'
          install-extras: tests
          os: ubuntu-24.04-arm

The wheel path detection logic changed, but I think that's ok. Would be nice if xcookie could keep the comments in the CI yaml. I was having issues getting ruamel.yaml doing that. I may manually update if I can solve the CI issues.

[TTsangSC]

Some observations:

• The common failure points on the jobs seem to consistently be tests/test_child_procs.py::test_running_multiproc_script, i.e. the tests created in FIX: kernprof to always run code in sys.modules['__main__']'s namespace #423 to guard against BUG: kernprof executed code inconsistent with sys.modules['__main__'], causing pickling issues #422. Which is strange – after all, the PR which introduced the tests also implemented the fix. The problem seems to be that somehow pip install isn't respecting the -f flag, pulling the wheel from PyPI instead of ./wheelhouse.

  • Given that the wheelhouse and a wheel of the matching version and architecture exist, IDK why this would happen... can't seem to reproduce this behavior on local with (vanilla) pipeither.
  • Unfortunately the equivalent uv command in the passing pipeline didn't produce enough output to indicate whence it pulled the line_profiler wheel, but I have to guess the wheelhouse and not PyPI judging from how the tests did pass.

• The 3.9 [...] with tests-strict,runtime-strict jobs additionally fail the ref-counting in these four tests in tests/test_line_profiler.py:

  • test_double_decoration
  • test_function_decorator
  • test_async_gen_decorator[...]

  The cause is how the tests-strict extra pins coverage to 6.5 (see requirements/tests.txt); in my testing, anything below 7.3 consistently breaks these tests between Python 3.9 and 3.11. This bug seems consistent with what is described in CTracer fails to deallocate bound methods references coveragepy/coveragepy#1283 (reported on Python 3.8, coverage 6.1), where a couple refs are leaked when coverage is used; this however begs the question of why this adverse interaction didn't manifest when we had equivalent setups on Python 3.8? (Same discrepancy on local – 6.5 works with 3.8 but not any newer Pythons.)

• Looking at the diffs, not only is the ARM-Linux stuff gone (regressing FIX: building Linux-ARM64 wheels #402), one of the test jobs on regular Ubuntu (3.13 on ubuntu-latest, arch=auto with tests) is also caught in the crossfire:

           - python-version: '3.13'
             install-extras: tests-strict,runtime-strict,optional-strict
             os: windows-11-arm
             arch: auto
  -        - python-version: '3.13'
  -          install-extras: tests
  -          os: ubuntu-latest
  -          arch: auto
  -        - python-version: '3.13'
  -          install-extras: tests
  -          os: ubuntu-24.04-arm
  -          arch: auto
           - python-version: '3.13'
             install-extras: tests
             os: macOS-latest
             arch: auto

• Is there any reason why we're removing cp39-win_arm64 from cibw_skip?

  -        - '*-win32 cp3{9,10}-win_arm64 cp313-musllinux_i686'
  +        - '*-win32 cp3{10}-win_arm64 cp313-musllinux_i686'

  As described in FIX: building on Windows-ARM64 #391, we skipped these Python versions because we can't test them on GitHub's windows-11-arm runners. As far as I can tell, this constraint is still true:

  test script

  import json
  import subprocess  # I don't have requests
  
  
  # Format:
  # [{"version": ...,
  #   "stable": ...,
  #   "release_url": ...,
  #   "files": [{"filename": ...,
  #               "arch": ...,
  #               "platform": ...,
  #               "platform_version": ...,  # Optional
  #               "download_url": ...},
  #             ...]},
  #  ...]
  manifest = 'https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json'
  proc = subprocess.run(['curl', manifest], text=True, capture_output=True)
  versions = json.loads(proc.stdout)
  
  
  def is_available(python_version: str) -> bool:
      return any((version['version'].startswith(python_version)
                  and any((file['arch'].startswith('arm64')
                           and file['platform'] == 'win32')
                          for file in version['files']))
                 for version in versions)
  
  
  assert is_available('3.11')
  assert not is_available('3.10')
  assert not is_available('3.9')

  Not to mention: even if the intent was to specifically un-skip building 3.9 wheel for ARM-Windows, this form still causes issues as braces without multiple options therein isn't correctly parsed by cibuildwheel.

[Erotemic]

@TTsangSC I've bumped the minimum versions of coverage / pytest-cov to prevent the failures, and I'm working on adding things back in so I can squash all these patches into a reasonable diff that will fix your PRs.

Thanks for the analysis, it was very helpful.

EDIT: Also noting, I want to add back in the removal of qemu given that we are running the CI's on arm naively.

[Erotemic]

I'm thinking we just EOL Python 3.9 while we are at it, and just keep up with the non-EOL python versions. I'm also going to switch over to github trusted publishing to reduce long-lived secrets.

[TTsangSC]

Hi, sorry for the radio silence the last week or so, was working on stuff.

As of 714c414 we're still missing

- python-version: '3.13'
  install-extras: tests
  os: ubuntu-latest
  arch: auto

While the platform is already well-covered by the other test jobs, I think we should add this back for completeness, given that there is an equivalent job for windows-11-arm, macOS-latest, windows-latest, and ubuntu-24.04-arm.

[TTsangSC]

Yeah, cibuildwheel can't deal with braces without comma-separated options:

Warning: cibuildwheel: Invalid skip selector: 'cp3{10}-win_arm64'. This selector will have no effect. 

We should probably either drop the skip or fix the label (-> cp310-win_arm64).

[TTsangSC]

Ditto the comment for .github/workflows/tests.yml.

[TTsangSC]

It's interesting that the pipelines for #430 went through without your first merging this... guess that GitHub got their stuff together and dialed back the rate limiting for the files that legacy cibuildwheel versions pull.

[Erotemic]

Yeah, I was expecting the same error and was confused when I didn't see it. I'm wondering if rerunning the CI on your branches will give you a green status. I didn't see the normal way to do that. Either I can't do it for commits from other people or the CI run is too old to rerun. If you want to try you can push up a dummy commit, otherwise I hope to land this within the week.

FWIW, one of the reasons I've been taking so long on this is because I wanted to get trusted publishing setup and improve the security posture a bit. This separates out all the secret related stuff into a release.yml that is different from the normal test.yml

[TTsangSC]

I guess it's just that my PRs have "gone stale" by GitHub's standard, pretty sure you have re-triggered pipelines on them before, which however stayed red because of the reasons this PR was addressing.

Thanks for the update, I'll check if there are stray typos in the PRs and fix them, or just do dummy pushes otherwise.

(There's also a big new feature PR coming... should be ready in half an hour. ;))

EDIT: time estimates were wrong, classic me. Anyway #431 is now online.

[Erotemic]

@TTsangSC rebasing on main should fix your branches now.