Skip to content

Invalid SHELL in Makefile for AIX builds? #149021

Open
Open
Invalid SHELL in Makefile for AIX builds?#149021
Labels
OS-unsupportedbuildThe build process and cross-buildtype-bugAn unexpected behavior, bug, or error

Description

@sideeffect42
sideeffect42
opened on Apr 26, 2026

Bug report

Bug description:

When trying to build CPython on AIX, the make steps fails immediately with the following error message:

$ ./configure
[...]
$ make       
/bin/sh -e: not found

make: 1254-004 The error code from the last command is 1.


Stop.

AIX make(1) interprets whatever is assigned to the SHELL variable as a file path to the shell. It thus tries to use a shell named "/bin/sh -e":

execve("/bin/sh -e", 0x0000000110004D28, 0x0FFFFFFFFFFFFB90) Err#2  ENOENT

The -e was added by #100220, however I wonder if this is the right approach.

Going by POSIX I think -e should be the default unless it is explicitly disabled:

The execution line shall then be executed by a shell as if it were passed as the argument to the system() interface, except that if errors are not being ignored then the shell -e option shall also be in effect. If errors are being ignored for the command (as a result of the -i option, a '-' command prefix, or a .IGNORE special target), the shell -e option shall not be in effect.

I assume the -e was added for GNU make, which does not set -e unless it is set to POSIX-conforming mode (GNU make: Choosing the shell).

If the Makefile is defined to be a POSIX Makefile, wouldn't it be better to define .POSIX: in the makefile?

Quick test with GNU make:

$ cat Makefile
.POSIX:

SHELL = /bin/sh

test:
	false; echo after false
$ make --version
GNU Make 4.4.1
Built for powerpc64le-unknown-linux-musl
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ make test
false; echo after false
make: *** [Makefile:4: test] Error 1

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs

Activity

  1. sideeffect42
    added
    type-bugAn unexpected behavior, bug, or error
    on Apr 26, 2026
  2. StanFromIreland
    added
    buildThe build process and cross-build
    OS-unsupported
    on Apr 26, 2026
  3. github-project-automation
    added this to Unsupported platformson Apr 26, 2026

  4. picnixz commented on Apr 26, 2026

    @picnixz
    picnixz
    on Apr 26, 2026
    Member
    1. We have AIX build bots so I'm surprised that this wasn't caught.
    2. This happens to me on non-AIX platforms sometimes because of a non-clean configure step or whatever. So just to be sure, your ./configure call was successful and was from a clean build right?
    3. I'm not entirely sure that we assume GNU Make actually. I know that because it's not GNU make we don't use wildcard targets or %-based targets as it's not portable. Is .POSIX portable?
    4. We could limit this change to AIX only with an additional configure step (where we detect the existing make + platform and decide how to include the -e option).

    @aisk I think you have access to AIX, can you check please? (if not sorry for the ping)

  5. picnixz
    changed the title [-]Invalid `SHELL` in `Makefile`?[/-] [+]Invalid `SHELL` in `Makefile` for AIX builds?[/+] on Apr 26, 2026

  6. sideeffect42 commented on Apr 26, 2026

    @sideeffect42
    sideeffect42
    on Apr 26, 2026
    Author

    @picnixz, thank you for looking at this issue so quickly.

    1. Looking at some buildbot logs it seems to me that the AIX builders use GNU make.
      GNU make can be installed on AIX from the so called "Linux Toolbox".

    2. It happens every time and after I patch the SHELL variable (and apply another tiny Makefile patch) manually, the build goes through successfully.

    3. I thought that Python could be assuming GNU make silently; that's why I opened an issue first and didn't submit a PR already.
      It would be a shame though, because the Makefile looks like it works with other makes fine otherwise.

      I would consider .POSIX: portable, it's defined by the POSIX standard afterall.
      I tested my little shell snippet successfully with /usr/bin/make on AIX, FreeBSD, OpenBSD.
      NetBSD prints after false no matter if .POSIX and/or SHELL = /bin/sh -e is defined and requires a special .SHELL target to configure the shell.

      The only thing I could see possibly making problems would be Windows.

    4. Depending on how one interprets POSIX, one could make the point that the SHELL variable is meant to only contain a path and adding options is non-standard:

      The SHELL macro shall be treated specially. It shall be provided by make and set to the pathname of the shell command language interpreter (see sh). The SHELL environment variable shall not affect the value of the SHELL macro. If SHELL is defined in the makefile or is specified on the command line, it shall replace the original value of the SHELL macro, but shall not affect the SHELL environment variable. Other effects of defining SHELL in the makefile or on the command line are implementation-defined.

      If GNU make and NetBSD do not feel like conforming to the standard (the -e was added due to some issues on Gentoo Linux popping up which uses GNU make), I think the configure checks should be the other way round.

  7. picnixz commented on Apr 27, 2026

    @picnixz
    picnixz
    on Apr 27, 2026
    Member

    Looking at some buildbot logs it seems to me that the AIX builders use GNU make.
    GNU make can be installed on AIX from the so called "Linux Toolbox".

    Just to be sure, but the original failure you experienced is because you are using non-GNU make, right?

    I thought that Python could be assuming GNU make silently

    GNU Make has slightly more extensions than plain make and AFAIR, we assume plain make to ensure maximum compatibility. If .POSIX is defined in the standards, then it might indeed be better to do it.

    The only thing I could see possibly making problems would be Windows.

    You don't need to worry about it, we don't use that Makefile on Windows I think (?) (cc @chris-eibl)

    NetBSD prints after false no matter if .POSIX and/or SHELL = /bin/sh -e is defined and requires a special .SHELL target to configure the shell.

    So this actually never properly worked on NetBSD, hum.

    I think the configure checks should be the other way round

    Sure, whichever way is cleaner and doesn't break others would be good!

    So, we have at least two issues at hand:

    • Make sure that NetBSD's make correctly works as expected (the use of our SHELL is incorrect for them).
    • Make sure that it works with plain make and GNU Make in AIX and other non-PEP-11 platforms. For that, we need to determine how the plain Make of each platforms works. I think we can have a default assumption where the current Makefile works. As such, we need some templating in Makefile.pre.in where the way we invoke the SHELL is a template that we fill during configure.

  8. chris-eibl commented on Apr 27, 2026

    @chris-eibl
    chris-eibl
    on Apr 27, 2026
    Member
    1. We have AIX build bots so I'm surprised that this wasn't caught.

    We do not have any AIX workers anymore: python/buildmaster-config#663 (review)

    Since the change removes the last AIX workers, I suggest removing also the factories: AIXBuild and AIXBuildWithXLC.

    You don't need to worry about it, we don't use that Makefile on Windows I think (?) (cc @chris-eibl)

    👍

  9. sideeffect42 commented on Apr 27, 2026

    @sideeffect42
    sideeffect42
    on Apr 27, 2026
    Author

    Just to be sure, but the original failure you experienced is because you are using non-GNU make, right?

    Correct, I'm using AIX's "own" /usr/bin/make.

    GNU make (when installed from the toolbox) would be /opt/freeware/bin/gmake.

    I thought that Python could be assuming GNU make silently

    GNU Make has slightly more extensions than plain make and AFAIR, we assume plain make to ensure maximum compatibility. If .POSIX is defined in the standards, then it might indeed be better to do it.

    OK, thanks for clarifying. In this case I think adding .POSIX: is a good thing also to make it clear to developers that this is a POSIX Makefile, not a GNU/BSD/whatever makefile.

    Though, then I'm spotting some other things which I'm pretty sure are not POSIX compliant, e.g.

    * PY_CORE_EXE_LDFLAGS:= $(if $(CONFIGURE_EXE_LDFLAGS), $(CONFIGURE_EXE_LDFLAGS) $(PY_LDFLAGS_NODIST), $(PY_CORE_LDFLAGS
))

$(shell date +%s)
$(shell $(GITVERSION))

$(filter-out $(JIT_SHIM_BUILD_OBJS),$(JIT_OBJS))

    All of these make FreeBSD /usr/bin/make complain:

    make: /home/user/cpython/Makefile:132: warning: Invalid character " " in variable name "if ,   $(LDFLAGS_NODIST),    $(LDFLAGS_NODIST)"
make: /home/user/cpython/Makefile:2529: warning: Invalid character " " in variable name "shell date +%s"
make: /home/user/cpython/Makefile:3398: warning: Invalid character " " in variable name "filter-out ,"

    AIX make just converts these expansions to empty strings.

    If you just ignore those errors, you'll however get a working ./python as a result.

    Another thing I'm mentioning just for completeness is that if Python relies on a specific set of default rules, adding .POSIX may make make choose a different set of default rules.
    But I'm pretty sure the default set differs between the different implementations as well.

    NetBSD prints after false no matter if .POSIX and/or SHELL = /bin/sh -e is defined and requires a special .SHELL target to configure the shell.

    So this actually never properly worked on NetBSD, hum.

    I mean I haven't actually tested a CPython build on NetBSD.
    I'd assume it (could) work, it will just not to error handling as it should.

    That's assuming people use /usr/bin/make (~bmake).
    Of course NetBSD has GNU make in pkgsrc as well.

    I think the configure checks should be the other way round

    Sure, whichever way is cleaner and doesn't break others would be good!

    So, we have at least two issues at hand:

    • Make sure that NetBSD's make correctly works as expected (the use of our SHELL is incorrect for them).
    • Make sure that it works with plain make and GNU Make in AIX and other non-PEP-11 platforms. For that, we need to determine how the plain Make of each platforms works. I think we can have a default assumption where the current Makefile works. As such, we need some templating in Makefile.pre.in where the way we invoke the SHELL is a template that we fill during configure.

    Agreed, and remove the few GNU-isms which did make it into the Makefile; so far I could spot 4 of them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-unsupportedbuildThe build process and cross-buildtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions