Skip to content

gh-113785: csv: fields starting with escapechar are not quoted#122110

Merged
serhiy-storchaka merged 4 commits into
python:mainfrom
MKuranowski:fix-issue-113785
Jul 25, 2024
Merged

gh-113785: csv: fields starting with escapechar are not quoted#122110
serhiy-storchaka merged 4 commits into
python:mainfrom
MKuranowski:fix-issue-113785

Conversation

@MKuranowski

@MKuranowski MKuranowski commented Jul 22, 2024

Copy link
Copy Markdown
Contributor

This PR fixes an inconsistency in the _csv.c module, whereby escapechars at the beginning of the field imply that fields are quoted, but escape characters at any other position do not. This only really affects parsing with QUOTE_NONNUMERIC - fields starting with the escape character are incorrectly left as a string, instead of being parsed.

With this change escapechars do not imply a quoted field; which allows fields like "\\.5" with QUOTE_NONNUMERIC to be correctly parsed as 0.5.

Closes #113785.

@ghost

ghost commented Jul 22, 2024

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app

bedevere-app Bot commented Jul 22, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Comment thread Lib/test/test_csv.py
@@ -0,0 +1 @@
:mod:`csv` now correctly parses numeric fields (when used with :const:`csv.QUOTE_NONNUMERIC`) which start with an escape character.

@serhiy-storchaka serhiy-storchaka Jul 23, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or csv.QUOTE_STRINGS.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in cf05e97

@serhiy-storchaka serhiy-storchaka added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Jul 23, 2024

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@serhiy-storchaka serhiy-storchaka merged commit a3327db into python:main Jul 25, 2024
@miss-islington-app

Copy link
Copy Markdown

Thanks @MKuranowski for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 25, 2024
…ythonGH-122110)

(cherry picked from commit a3327db)

Co-authored-by: Mikołaj Kuranowski <mkuranowski@gmail.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 25, 2024
…ythonGH-122110)

(cherry picked from commit a3327db)

Co-authored-by: Mikołaj Kuranowski <mkuranowski@gmail.com>
@bedevere-app

bedevere-app Bot commented Jul 25, 2024

Copy link
Copy Markdown

GH-122258 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Jul 25, 2024
@bedevere-app

bedevere-app Bot commented Jul 25, 2024

Copy link
Copy Markdown

GH-122259 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Jul 25, 2024
@serhiy-storchaka

Copy link
Copy Markdown
Member

Thank you for your contribution @MKuranowski.

serhiy-storchaka pushed a commit that referenced this pull request Jul 25, 2024
…GH-122110) (GH-122258)

(cherry picked from commit a3327db)

Co-authored-by: Mikołaj Kuranowski <mkuranowski@gmail.com>
serhiy-storchaka pushed a commit that referenced this pull request Aug 9, 2024
…GH-122110) (GH-122259)

(cherry picked from commit a3327db)

Co-authored-by: Mikołaj Kuranowski <mkuranowski@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSV reader inconsistent with combination of QUOTE_NONNUMERIC and escapechar

2 participants