ValueError: source code string cannot contain null bytes #1499

q144 · 2021-10-16T18:53:42Z

q144
Oct 16, 2021

Description

hi,
i upgraded my working system to 4.18
i came from a working v4.16.3 (not 100% sure but i was approx. 3 versions behind)
i got a "Executing database migrations failed" error.
i reinstalled a recent copy of the DB, retried the update (deploy.sh) and i got same problem.
i downgraded to 4.16.3 (reinstalled that db copy) and it works fine again

any suggestions?
i can provide copy of my db if this would be preferable

note : when importing my db copy i got only 1 small issue but this seems to be not related at all

COPY 0
ERROR: null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null, null, null, null).
CONTEXT: COPY dsmr_consumption_gasconsumption, line 5998: "\N \N \N \N"
COPY 1

Log : upgrade output

--- Applying database migrations.
Traceback (most recent call last):
File "./manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/core/management/init.py", line 401, in execute_from_command_line
utility.execute()
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/core/management/init.py", line 377, in execute
django.setup()
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
app_config.ready()
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/admin/apps.py", line 24, in ready
self.module.autodiscover()
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/admin/init.py", line 24, in autodiscover
autodiscover_modules('admin', register_to=site)
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/auth/admin.py", line 6, in
from django.contrib.auth.forms import (
File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/auth/forms.py", line 12, in
from django.contrib.sites.shortcuts import get_current_site
ValueError: source code string cannot contain null bytes

                    [!] Executing database migrations failed! <<<<<
                    [i] Trying to automatically resolve with 'dsmr_sqlsequencereset'.
                    Traceback (most recent call last):
                    File "./manage.py", line 10, in
                    execute_from_command_line(sys.argv)
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/core/management/init.py", line 401, in execute_from_command_line
                    utility.execute()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/core/management/init.py", line 377, in execute
                    django.setup()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/init.py", line 24, in setup
                    apps.populate(settings.INSTALLED_APPS)
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
                    app_config.ready()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/admin/apps.py", line 24, in ready
                    self.module.autodiscover()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/admin/init.py", line 24, in autodiscover
                    autodiscover_modules('admin', register_to=site)
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
                    import_module('%s.%s' % (app_config.name, module_to_search))
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/importlib/init.py", line 127, in import_module
                    return _bootstrap._gcd_import(name[level:], package, level)
                    File "", line 1006, in _gcd_import
                    File "", line 983, in _find_and_load
                    File "", line 967, in _find_and_load_unlocked
                    File "", line 677, in _load_unlocked
                    File "", line 728, in exec_module
                    File "", line 219, in _call_with_frames_removed
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/auth/admin.py", line 6, in
                    from django.contrib.auth.forms import (
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/auth/forms.py", line 12, in
                    from django.contrib.sites.shortcuts import get_current_site
                    ValueError: source code string cannot contain null bytes
                    Traceback (most recent call last):
                    File "./manage.py", line 10, in
                    execute_from_command_line(sys.argv)
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/core/management/init.py", line 401, in execute_from_command_line
                    utility.execute()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/core/management/init.py", line 377, in execute
                    django.setup()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/init.py", line 24, in setup
                    apps.populate(settings.INSTALLED_APPS)
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
                    app_config.ready()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/admin/apps.py", line 24, in ready
                    self.module.autodiscover()
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/admin/init.py", line 24, in autodiscover
                    autodiscover_modules('admin', register_to=site)
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
                    import_module('%s.%s' % (app_config.name, module_to_search))
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/importlib/init.py", line 127, in import_module
                    return _bootstrap._gcd_import(name[level:], package, level)
                    File "", line 1006, in _gcd_import
                    File "", line 983, in _find_and_load
                    File "", line 967, in _find_and_load_unlocked
                    File "", line 677, in _load_unlocked
                    File "", line 728, in exec_module
                    File "", line 219, in _call_with_frames_removed
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/auth/admin.py", line 6, in
                    from django.contrib.auth.forms import (
                    File "/home/dsmr/.virtualenvs/dsmrreader/lib/python3.7/site-packages/django/contrib/auth/forms.py", line 12, in
                    from django.contrib.sites.shortcuts import get_current_site
                    ValueError: source code string cannot contain null bytes
                    [!] Executing database migrations failed again! <<<<<
                    - Running dsmr_sqlsequencereset did not resolve the problem.
                    - Create an issue on GitHub and attach the exeception trace listed above.
                    [!] Halted. Post-deployment script exited with non-zero code

DSMR-reader version

4.18

DSMR-reader platform

Native (e.g. manual installation)

Debug info dump

No response

Answered by dennissiemensma

Oct 18, 2021

I'm not sure which source file is broken, it may be one of the external packages, but not sure. You could try reinstalling the virtual env with external packages. Not sure if it helps though.

I found one other report last year: #925 (comment)
Which was fixed by it. You can give those commands a shot, but ensure to bump the code to the latest version first.

sudo supervisorctl stop all

sudo su - dsmr 

# Bump code so the new env below installs the latest one
git checkout v4
git pull

rm -rf ~/.virtualenvs/dsmrreader/
virtualenv /home/dsmr/.virtualenvs/dsmrreader --no-site-packages --python python3
/home/dsmr/.virtualenvs/dsmrreader/bin/pip3 install -r /home/dsmr/dsmr-reader/dsmrreader/prov…

View full answer

dennissiemensma · 2021-10-16T20:36:11Z

dennissiemensma
Oct 16, 2021
Maintainer

Thanks for reporting, what OS are you running DSMR-reader on?

The ValueError: source code string cannot contain null bytes error may be caused by weird characters in your local copy of DSMR-reader, but I'm not sure what is causing it.

What is the output of:

sudo su - dsmr

git status
git diff

0 replies

q144 · 2021-10-16T21:36:51Z

q144
Oct 16, 2021
Author

(dsmrreader) dsmr@raspberrypi:~/dsmr-reader $ uname -a
Linux raspberrypi 5.10.17-v7+ #1403 SMP Mon Feb 22 11:29:51 GMT 2021 armv7l GNU/Linux

(dsmrreader) dsmr@raspberrypi:~/dsmr-reader $  cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"  ......

(dsmrreader) dsmr@raspberrypi:~/dsmr-reader $ git status
HEAD detached at v4.16.3
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.env.swp
nothing added to commit but untracked files present (use "git add" to track)

(dsmrreader) dsmr@raspberrypi:~/dsmr-reader $ git diff
(dsmrreader) dsmr@raspberrypi:~/dsmr-reader $

i could (re)test with a specific build if you could provide the git commands
as im not familiar how to switch between the builds or even go back to main

0 replies

dennissiemensma · 2021-10-18T19:03:45Z

dennissiemensma
Oct 18, 2021
Maintainer

I'm not sure which source file is broken, it may be one of the external packages, but not sure. You could try reinstalling the virtual env with external packages. Not sure if it helps though.

I found one other report last year: #925 (comment)
Which was fixed by it. You can give those commands a shot, but ensure to bump the code to the latest version first.

sudo supervisorctl stop all

sudo su - dsmr 

# Bump code so the new env below installs the latest one
git checkout v4
git pull

rm -rf ~/.virtualenvs/dsmrreader/
virtualenv /home/dsmr/.virtualenvs/dsmrreader --no-site-packages --python python3
/home/dsmr/.virtualenvs/dsmrreader/bin/pip3 install -r /home/dsmr/dsmr-reader/dsmrreader/provisioning/requirements/base.txt

Then try again:

logout
sudo su - dsmr
./deploy.sh

0 replies

dennissiemensma · 2021-10-18T19:04:52Z

dennissiemensma
Oct 18, 2021
Maintainer

Moved issue, you can still comment/update at this discussion now @q144

0 replies

q144 · 2021-10-20T20:10:24Z

q144
Oct 20, 2021
Author

thanks,
it works now,
i had to logout from dsmr for some parts to get it done .

note that this file is missing from the 4.8 tree.
/home/dsmr/dsmr-reader/dsmrreader/provisioning/requirements/postgresql.txt
i used the copy from another tree ("psycopg2-binary==2.8.4") to fix this

0 replies

dennissiemensma · 2021-10-20T20:29:13Z

dennissiemensma
Oct 20, 2021
Maintainer

Great, thanks for trying. I'll update the commands above. The file you were missing was removed the past year, hence the error.

You can either do:

sudo apt-get install -y python3-psycopg2

Before recreating the virtualenv, or install psycopg2-binary, like you did. Both should work indeed!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ValueError: source code string cannot contain null bytes #1499

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 6 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

ValueError: source code string cannot contain null bytes #1499

q144 Oct 16, 2021

Description

DSMR-reader version

DSMR-reader platform

Debug info dump

Replies: 6 comments

dennissiemensma Oct 16, 2021 Maintainer

q144 Oct 16, 2021 Author

dennissiemensma Oct 18, 2021 Maintainer

dennissiemensma Oct 18, 2021 Maintainer

q144 Oct 20, 2021 Author

dennissiemensma Oct 20, 2021 Maintainer

q144
Oct 16, 2021

dennissiemensma
Oct 16, 2021
Maintainer

q144
Oct 16, 2021
Author

dennissiemensma
Oct 18, 2021
Maintainer

dennissiemensma
Oct 18, 2021
Maintainer

q144
Oct 20, 2021
Author

dennissiemensma
Oct 20, 2021
Maintainer