Skip to content

Fix ordinal day parsing in locales without suffixes - #1091

Open
CyberKun wants to merge 2 commits into
python-pendulum:masterfrom
CyberKun:fix/ordinal-parser-errors
Open

CyberKun wants to merge 2 commits into
python-pendulum:masterfrom
CyberKun:fix/ordinal-parser-errors

Conversation

@CyberKun

Copy link
Copy Markdown

from_format() crashes with AttributeError for Do in locales without custom ordinal suffixes, even when parsing the output of format(). For example, a German YYYY-MM-Do string cannot round-trip.

Use a bare numeric pattern when the locale has no ordinal suffix mapping, and escape existing suffixes as literal text. Also reject localized tokens mapped to None with the existing ValueError("Unsupported token: ...") diagnostic instead of looking up a nonexistent translation.

Partially addresses #1074: this fixes Do and the unsupported-token errors. Adding parsing support for Mo, Qo, and DDDo remains separate work.

Validation:

  • 62 parametrized regression cases passed (9 locales across 6 days, plus 8 unsupported tokens). Before the fix, 50 failed and 12 passed.

  • Existing formatting tests: 22 passed.

  • Ruff 0.14.11 lint and format checks passed for both changed files.

  • Tests ran on Python 3.14 with PYTHONPATH=src and PENDULUM_EXTENSIONS=0. The full suite was not run: Windows Application Control blocks the native time-machine dependency used by the time-travel tests.

  • Added tests for changed code.

  • Updated documentation for changed code (no new supported tokens or public API).

Signed-off-by: Aditya <adityasbakawat@gmail.com>
Comment thread src/pendulum/formatting/formatter.py Outdated
Comment on lines 79 to 82
"Do": lambda locale: tuple(
rf"\d+{o}" for o in locale.get("custom.ordinal").values()
rf"\d+{re.escape(o)}"
for o in (locale.get("custom.ordinal") or {"other": ""}).values()
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Italian still can't round-trip a few days. it has custom.ordinal = {"other": "°"} but its plural rule gives many for 8, 11, 80, 800, and Locale.ordinalize returns the bare number when the category has no suffix. So format() writes 8 / 11, and the only candidate here is \d+°:

d = pendulum.datetime(2024, 1, 8)
text = d.format("YYYY-MM-Do", locale="it")  # '2024-01-8'
pendulum.from_format(text, "YYYY-MM-Do", locale="it")
# ValueError: String does not match format YYYY-MM-Do

Since ordinalize can always fall back to the bare number, adding \d+ as the last candidate covers both this and the no-mapping case (the suffix candidates come first in the alternation, so 1st / 1er still match whole):

Suggested change
"Do": lambda locale: tuple(
rf"\d+{o}" for o in locale.get("custom.ordinal").values()
rf"\d+{re.escape(o)}"
for o in (locale.get("custom.ordinal") or {"other": ""}).values()
),
"Do": lambda locale: (
*(
rf"\d+{re.escape(o)}"
for o in (locale.get("custom.ordinal") or {}).values()
if o
),
r"\d+",
),

I checked format → from_format for all 30 bundled locales, days 1–31, with YYYY-MM-Do, Do MMMM YYYY, MMMM Do, YYYY and Do.MM.YYYY: with the current branch only it fails (days 8 and 11); with the change above 0 failures, and tests/datetime/test_from_format.py + tests/formatting still pass (145). Adding "it" to the locale list in your round-trip test would catch it.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, 3091abe fixes it. I ran a format → parse round trip for all 30 bundled locales, days 1-31, in 5 formats (YYYY-MM-Do, Do MMMM YYYY, MMMM Do, YYYY, Do.MM.YYYY, Do MMM YYYY): 0 failures, Italian 8/11 included. The re.escape on the suffix is a good catch too.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants