Skip to content
This repository was archived by the owner on May 10, 2023. It is now read-only.

Conversation

tsabi
Copy link

@tsabi tsabi commented Dec 1, 2019

If you use postfix internationally, you will hit this error.
I believe this is the right fix, because those log files are NOT utf-8 encoded,
so you should not read it as utf-8, but with the real encoding of that file.

Fixes: modoboa/modoboa#1788, #55, #38,

root@h7-mailsrv:/etc/cron.d# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser
Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 587, in handle
    p.process()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 552, in process
    for line in self.f.readlines():
  File "/srv/modoboa/env/lib/python2.7/codecs.py", line 314, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc1 in position 949: invalid start byte

If you use postfix internationally, you will hit this error.
I believe this is the right fix, because those log files are NOT utf-8 encoded,
so you should not read it as utf-8, but with the real encoding of that file.

Fixes: modoboa/modoboa#1788, modoboa#55, modoboa#38, 

```
root@h7-mailsrv:/etc/cron.d# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser
Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 587, in handle
    p.process()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 552, in process
    for line in self.f.readlines():
  File "/srv/modoboa/env/lib/python2.7/codecs.py", line 314, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc1 in position 949: invalid start byte
```
@mundschenk-at
Copy link
Contributor

I got a similar error tonight: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 2515: invalid continuation byte. The rought cause in the logfile appears to be an issue with an invalid domain name.

postfix/trivial-rewrite[49231]: warning: midna_domain_to_ascii_create: Problem translating domain [...] to ASCII form: UIDNA_ERROR_DISALLOWED

I don't think opening logfiles as latin1 is the correct fix, but the logparser should handle unicode exceptions.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UnicodeDecodeError during stat generation

2 participants