Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Locale issue #435

Closed
bouweandela opened this issue Jul 17, 2024 · 1 comment · Fixed by #436
Closed

Locale issue #435

bouweandela opened this issue Jul 17, 2024 · 1 comment · Fixed by #436
Labels
community New: Issue Highlight a new community raised "generic" issue Type: Bug

Comments

@bouweandela
Copy link
Member

bouweandela commented Jul 17, 2024 •
edited
Loading

🐛 Bug Report

cf_units fails to import when the LC_NUMERIC environmental variable is set to a locale that has a , instead of a . as the decimal_point character and Python sets the locale. This typically happens in documentation builds with sphinx. See pydata/xarray#4257 for an example. I have encountered the same issue when building the iris and ESMValCore documentation.

How to Reproduce

Steps to reproduce the behaviour:

  1. Run
    export LC_NUMERIC=nl_NL.UTF-8
  2. View locale settings by running locale:
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=nl_NL.UTF-8
LC_TIME=nl_NL.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=nl_NL.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=nl_NL.UTF-8
LC_NAME=nl_NL.UTF-8
LC_ADDRESS=nl_NL.UTF-8
LC_TELEPHONE=nl_NL.UTF-8
LC_MEASUREMENT=nl_NL.UTF-8
LC_IDENTIFICATION=nl_NL.UTF-8
LC_ALL=

Run the following Python code to set the locale

import locale
locale.setlocale(locale.LC_ALL, '')
import cf_units
  1. Resulting stack trace:
Traceback (most recent call last):
  File "/home/bandela/src/scitools/cf_units/cf_units/__init__.py", line 187, in <module>
    _ud_system = _ud.read_xml()
                 ^^^^^^^^^^^^^^
  File "cf_units/_udunits2.pyx", line 194, in cf_units._udunits2.read_xml
    return wrap_system(csystem)
  File "cf_units/_udunits2.pyx", line 104, in cf_units._udunits2.wrap_system
    _raise_error()
  File "cf_units/_udunits2.pyx", line 184, in cf_units._udunits2._raise_error
    raise UdunitsError(status, errnum)
cf_units._udunits2.UdunitsError: UT_PARSE

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bandela/src/scitools/cf_units/cf_units/__init__.py", line 190, in <module>
    _ud_system = _ud.read_xml(config.get_xml_path())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cf_units/_udunits2.pyx", line 194, in cf_units._udunits2.read_xml
    return wrap_system(csystem)
  File "cf_units/_udunits2.pyx", line 104, in cf_units._udunits2.wrap_system
    _raise_error()
  File "cf_units/_udunits2.pyx", line 184, in cf_units._udunits2._raise_error
    raise UdunitsError(status, errnum)
cf_units._udunits2.UdunitsError: UT_PARSE

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bandela/src/scitools/cf_units/test.py", line 4, in <module>
    import cf_units
  File "/home/bandela/src/scitools/cf_units/cf_units/__init__.py", line 193, in <module>
    raise OSError(
OSError: [UT_PARSE] Failed to open UDUNITS-2 XML unit database

Expected Behaviour

A successful import.

Environment

  • OS & Version: Ubuntu 24.04 LTS
  • cf-units Version: 3.2.1.dev57

Additional Context

When I comment out the suppress_warnings context manager here

cf-units/cf_units/__init__.py

Line 182 in f57a7f1

with suppress_errors():

I see the following error message:

Invalid numeric prefix value ".1"
parsing aborted
File "/home/bandela/mambaforge/envs/cf-units-dev/share/udunits/udunits2-prefixes.xml", line 43, column 25
parsing aborted
File "/home/bandela/mambaforge/envs/cf-units-dev/share/udunits/udunits2.xml", line 11, column 42
Traceback (most recent call last):
  File "/home/bandela/src/scitools/cf_units/test.py", line 4, in <module>
    import cf_units
  File "/home/bandela/src/scitools/cf_units/cf_units/__init__.py", line 187, in <module>
    _ud_system = _ud.read_xml()
                 ^^^^^^^^^^^^^^
  File "cf_units/_udunits2.pyx", line 194, in cf_units._udunits2.read_xml
    return wrap_system(csystem)
  File "cf_units/_udunits2.pyx", line 104, in cf_units._udunits2.wrap_system
    _raise_error()
  File "cf_units/_udunits2.pyx", line 184, in cf_units._udunits2._raise_error
    raise UdunitsError(status, errnum)
cf_units._udunits2.UdunitsError: UT_PARSE

some searching trough the udunits2 code suggests that it originates from using the strtod function in this code, which uses the decimal-point locale setting.
@bouweandela bouweandela added New: Issue Highlight a new community raised "generic" issue Type: Bug labels Jul 17, 2024
This was referenced Jul 17, 2024
Closed
Merged
@bouweandela
Copy link
Member Author

I had a go at fixing this in #436, but there appear to be some issues with CI.

@valeriupredoi valeriupredoi mentioned this issue Sep 25, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
community New: Issue Highlight a new community raised "generic" issue Type: Bug
Projects
🚴 Peloton
Status: Done
Autumn 2024 cf-units
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Avoid issues with LC_NUMERIC locale bouweandela/cf-units
2 participants
@pp-mo @bouweandela