Skip to content

gh-125118: don't copy arbitrary values to _Bool in the struct module#125169

Merged
encukou merged 6 commits into
python:mainfrom
skirpichev:nu_bool-125118
Oct 10, 2024
Merged

gh-125118: don't copy arbitrary values to _Bool in the struct module#125169
encukou merged 6 commits into
python:mainfrom
skirpichev:nu_bool-125118

Conversation

@skirpichev

@skirpichev skirpichev commented Oct 9, 2024

Copy link
Copy Markdown
Member

memcopy'ing arbitrary values to _Bool variable triggers undefined behaviour, e.g.:

/* a.c */
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
    _Bool x;
    char y = 2;
    memcpy(&x, &y, 1);
    printf("%d\n", x != 0);
    return 0;
}
$ gcc-12 a.c -Wall && ./a.out
2
$ clang-15 a.c -Wall && ./a.out
0
$ gcc-12 a.c -Wall -fsanitize=undefined && ./a.out
a.c:9:5: runtime error: load of value 2, which is not a valid value for type '_Bool'
0
$ clang-15 a.c -Wall -fsanitize=undefined && ./a.out
a.c:9:20: runtime error: load of value 2, which is not a valid value for type '_Bool'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior a.c:9:20 in
0

Cridits to Alex Gaynor.

…odule

memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour, e.g.:
```c
/* a.c */
int main(void)
{
    _Bool x;
    char y = 2;
    memcpy(&x, &y, 1);
    printf("%d\n", x != 0);
    return 0;
}
```
```
$ gcc-12 a.c -Wall && ./a.out
2
$ clang-15 a.c -Wall && ./a.out
0
$ gcc-12 a.c -Wall -fsanitize=undefined && ./a.out
a.c:9:5: runtime error: load of value 2, which is not a valid value for type '_Bool'
0
$ clang-15 a.c -Wall -fsanitize=undefined && ./a.out
a.c:9:20: runtime error: load of value 2, which is not a valid value for type '_Bool'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior a.c:9:20 in
0
```

Cridits to Alex Gaynor.

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I would prefer to have a second review on this change.

@encukou: You worked on a similar issue some time ago, would you mind to review this change?

@encukou

encukou commented Oct 9, 2024

Copy link
Copy Markdown
Member

Yes, please wait for my review. I'll get to it this week.

@ZeroIntensity ZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks fine. Out of curiosity, what bug was this causing in practice?

@alex

alex commented Oct 9, 2024

Copy link
Copy Markdown
Member

It was triggering UBSAN warnings, because it was UB. AFAIK it wasn't misbehaving on any platform yet, but it'd have been within the compiler's right to rewrite != 0 as == 1, and that would have changed behavior.

Should we add a static_assert(sizeof(_Bool) == 1);? Otherwise LGTM.

@skirpichev

Copy link
Copy Markdown
Member Author

@ZeroIntensity, you could try added tests on gcc-12 (PASS) and clang-17 (FAIL). I suspect specific versions aren't important.

Comment thread Modules/_struct.c Outdated
@encukou

encukou commented Oct 9, 2024

Copy link
Copy Markdown
Member

As far as I can see, the safest assumption we can make is that false is all zero bytes.

To allow bigger sizeof(_Bool), that would be:

    static const _Bool false_bool = 0;
    return PyBool_FromLong(memcmp(p, &false_bool, sizeof(_Bool)));

@vstinner

vstinner commented Oct 9, 2024

Copy link
Copy Markdown
Member

static const _Bool false_bool = 0;

I don't think that static is needed here.

Co-authored-by: Sam Gross <colesbury@gmail.com>
@skirpichev skirpichev requested a review from encukou October 9, 2024 15:09
Comment thread Modules/_struct.c Outdated
Comment thread Modules/_struct.c Outdated
Comment thread Modules/_struct.c Outdated
Co-authored-by: Victor Stinner <vstinner@python.org>

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Comment thread Modules/_struct.c Outdated
Co-authored-by: Petr Viktorin <encukou@gmail.com>
@encukou encukou merged commit 87d7315 into python:main Oct 10, 2024
@encukou

encukou commented Oct 10, 2024

Copy link
Copy Markdown
Member

Thank you!

@encukou encukou added the needs backport to 3.12 only security fixes label Oct 10, 2024
@encukou encukou added the needs backport to 3.13 bugs and security fixes label Oct 10, 2024
@miss-islington-app

Copy link
Copy Markdown

Thanks @skirpichev for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington-app

Copy link
Copy Markdown

Thanks @skirpichev for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @skirpichev and @encukou, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 87d7315ac57250046372b0d9ae4619ba619c8c87 3.13

@miss-islington-app

Copy link
Copy Markdown

Sorry, @skirpichev and @encukou, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 87d7315ac57250046372b0d9ae4619ba619c8c87 3.12

@skirpichev skirpichev deleted the nu_bool-125118 branch October 10, 2024 13:51
@skirpichev

Copy link
Copy Markdown
Member Author

Working on backports...

skirpichev added a commit to skirpichev/cpython that referenced this pull request Oct 10, 2024
…truct module (pythonGH-125169)

memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.

Credits to Alex Gaynor.

(cherry picked from commit 87d7315)

Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
@bedevere-app

bedevere-app Bot commented Oct 10, 2024

Copy link
Copy Markdown

GH-125263 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Oct 10, 2024
skirpichev added a commit to skirpichev/cpython that referenced this pull request Oct 10, 2024
…truct module (pythonGH-125169)

memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.

Credits to Alex Gaynor.

(cherry picked from commit 87d7315)

Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
@bedevere-app

bedevere-app Bot commented Oct 10, 2024

Copy link
Copy Markdown

GH-125265 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Oct 10, 2024
vstinner added a commit that referenced this pull request Oct 10, 2024
…module (GH-125169) (#125265)

memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.

Credits to Alex Gaynor.

(cherry picked from commit 87d7315)

Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
vstinner added a commit that referenced this pull request Oct 10, 2024
…module (GH-125169) (#125263)

memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.

Credits to Alex Gaynor.

(cherry picked from commit 87d7315)

Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants