My task is to populate my two existing City and Province model using two json files cities.json' and 'provinces.json.
Data in these files are as below.
provinces.json:
[
{ "model": "salesnetwork.provinces",
"id": "1",
"name": "EA"
}
]
cities.json:
[
{ "model": "salesnetwork.cities",
"id": "1",
"province_id": "1",
"name": "Tabriz"
}
]
Now I'm trying to poplulate my two models with these data. My models are as below.
class Provinces(models.Model):
name = models.CharField(max_length=255, db_collation="utf8mb3_unicode_ci")
class Meta:
db_table = "provinces"
class Cities(models.Model):
province = models.ForeignKey("Provinces", models.DO_NOTHING)
name = models.CharField(max_length=255, db_collation="utf8mb3_unicode_ci")
class Meta:
db_table = "cities"
(My bad for naming my model Cities instead of City and Provinces instead of 'Province'; I've created these models via inspectdb command and since I didn't know if changing these model names could cause problems or not, I decided to leave the names as they were).
When I try using command py .\manage.py loaddata provinces.json I get the following error:
Traceback (most recent call last):
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\serializers\json.py", line 69, in Deserializer
objects = json.loads(stream_or_string)
File "C:\Users\Vahid Moradi\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 335, in loads
raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\manage.py", line 22, in <module>
main()
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\__init__.py", line 446, in execute_from_command_line
utility.execute()
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\base.py", line 448, in execute
output = self.handle(*args, **options)
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\commands\loaddata.py", line 102, in handle
self.loaddata(fixture_labels)
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\commands\loaddata.py", line 163, in loaddata
self.load_label(fixture_label)
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\management\commands\loaddata.py", line 251, in load_label
for obj in objects:
File "D:\Projects\Navid Motor\Website\Django\NavidMotor.com\.venv\lib\site-packages\django\core\serializers\json.py", line 74, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture 'D:\Projects\Navid Motor\Website\Django\NavidMotor.com\provinces.json'::
and by the looks of it, this time the problem is json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0), which I have no idea how to resolve and didn't find any instruction on how to solve this issue.
I did study the following questions, but all were for problems relating json library inside .py files, and since my problem occurs in command line using manage.py commands, I couldn't apply those solutions for my problem.
Possible simillar questions, but could not use the answers given:
First
Second
Third
Try to paste json into new file in vscode and save it.
Errorjson.decoder.JSONDecodeError: Unexpected UTF-8 BOM because your json files have UTF8-BOM encoding:
https://www.coderedcorp.com/blog/how-to-dump-your-django-database-and-load-it-into-/
Related
Working on a fairly large/complex Django project with a team, we occasionally see runserver crash with ValueError: embedded null byte. We restart runserver and it's fine - either for a few minutes or a few days. We can detect no pattern to what causes the crashes (seems totally random). Fortunately it's only happening in local development, not on our servers, but I'm worried that it will bite us down the road.
The stack trace below does not point to any location in our code -- seems to come either from Django or the virtualenv itself.
Using Django 1.9.8, Python 3.5.0, on El Capitan.
I can't see any way to debug this. Theories?
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 54, in execute
super(Command, self).execute(*args, **options)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 93, in handle
self.run(**options)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 102, in run
autoreload.main(self.inner_run, None, options)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 333, in main
reloader(wrapped_main_func, args, kwargs)
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 299, in python_reloader
reloader_thread()
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 275, in reloader_thread
change = fn()
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 204, in code_changed
for filename in gen_filenames():
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 114, in gen_filenames
basedirs = [os.path.abspath(basedir) for basedir in basedirs
File "/path/to/virtualenvs/ourproj/lib/python3.5/site-packages/django/utils/autoreload.py", line 115, in <listcomp>
if os.path.isdir(basedir)]
File "/path/to/virtualenvs/ourproj/bin/../lib/python3.5/genericpath.py", line 42, in isdir
st = os.stat(s)
ValueError: embedded null byte
One of the AppConfig objects has null byte in its path attribute.
One of the LOCALE_PATHS has null byte.
One of the files is in the "wrong" encoding, so Django treats something has null byte (e. g. AppConfig.path).
One of the files or directories in project directory has null byte (\x00) in its name.
Other reason.
autoreload.py lines related to this issue. os.path.isdir() raises ValueError: embedded null byte if its argument has null byte, e. g. os.path.isdir('foo\x00bar').
You can try to edit autoreload.py, comment out these lines temporarily:
basedirs = [os.path.abspath(basedir) for basedir in basedirs
if os.path.isdir(basedir)]
and add this:
temp_basedirs = []
for basedir in basedirs:
try:
if os.path.isdir(basedir):
temp_basedirs.append(os.path.abspath(basedir))
except ValueError:
print(basedir)
raise
basedirs = temp_basedirs
I am building a website with django/python. The website database I am building contains books, and I am trying to relate Character objects to Event objects by defining their relationships in the fixture. I load my fixtures from .json files using
loaddata fixtures <file>
This works for my models that have no relationships, but when I try to load a fixture that contains many-to-many relationships (characters appear in many events, events contain many characters) I get the following :
ValueError: Problem installing fixture: < file path >: "< Character: Lanoree Brock > " needs to have a value for field "character" before this many-to-many relationship can be used.
There is no field "character" in my model for Character:
class Character(models.Model):
id = models.IntegerField(primary_key = True)
name = models.CharField(max_length = ml)
bio = models.TextField()
event = models.ManyToManyField(Event)
def __str__(self):
return self.name
class Meta:
ordering = ('name',)
The .json file for my Character fixture looks like this:
[{"model": "library.Character", "id": 1,
"fields": {"name": "Lanoree Brock", "bio": "He lived", "event": [101, 102, ...]}}
... ]
So the error occurs at the first Character. My guess as to the problem is that django tries to add the relationship to the < Character: Lanoree Brock > object before it saves the object, but I do not understand why it is doing that or how to get around it.
Is there a way to structure the fixture to ensure that when it is loaded each object is created/saved before the code tries to define its relationships?
I am new to django, JSON syntax, and web dev in general, and I feel like there's something simple I'm not doing here -- if fixtures cannot handle many-to-many relationships, that seems like a huge oversight in their functionality.
Any help would be appreciated, thank you!
Edit: The full error log:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\__init__.py", line 353, in execute_from_command_line
utility.execute()
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\base.py", line 399, in execute
output = self.handle(*args, **options)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\commands\loaddata.py", line 60, in handle
self.loaddata(fixture_labels)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\commands\loaddata.py", line 100, in loaddata
self.load_label(fixture_label)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\managemen
t\commands\loaddata.py", line 158, in load_label
obj.save(using=self.using)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\core\serialize
rs\base.py", line 204, in save
setattr(self.object, accessor_name, object_list)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\db\models\fiel
ds\related_descriptors.py", line 480, in __set__
manager = self.__get__(instance)
File "c:\Users\MetalGearSamus\Anaconda\lib\site-packages\django\db\models\fiel
ds\related_descriptors.py", line 468, in __get__
return self.related_manager_cls(instance)
File "c:Users\MetalGearSamus\Anaconda\lib\site-packages\django\db\models\fiel
ds\related_descriptors.py", line 751, in __init__
(instance, self.source_field_name))
ValueError: Problem installing fixture 'c:Users\MetalGearSamus\Personal\Legends\website\library\fixtures\database.json': "<Character: Lanoree Brock>" needs to
have a value for field "character" before this many-to-many relationship can be
used.
I'm working with Django 1.9 in Microsoft Visual Studio. Also I'm using a virtual env, where all the needed python packages are installed. When I try to compile messages in order to add translation function to the site, the following error is raised:
ValueError: plural forms expression could be dangerous
Update 1. Stacktrace:
Internal Server Error: /
Traceback (most recent call last):
File "C:\path-to-project\env\lib\site-packages\django\core\handlers\base.py", line 123, in get_response
response = middleware_method(request)
File "C:\path-to-project\env\lib\site-packages\django\middleware\locale.py", line 26, in process_request
translation.activate(language)
File "C:\path-to-project\env\lib\site-packages\django\utils\translation\__init__.py", line 154, in activate
return _trans.activate(language)
File "C:\path-to-project\env\lib\site-packages\django\utils\translation\trans_real.py", line 216, in activate
_active.value = translation(language)
File "C:\path-to-project\env\lib\site-packages\django\utils\translation\trans_real.py", line 205, in translation
_translations[language] = DjangoTranslation(language)
File "C:\path-to-project\env\lib\site-packages\django\utils\translation\trans_real.py", line 114, in __init__
self._add_installed_apps_translations()
File "C:\path-to-project\env\lib\site-packages\django\utils\translation\trans_real.py", line 160, in _add_installed_apps_translations
translation = self._new_gnu_trans(localedir)
File "C:\path-to-project\env\lib\site-packages\django\utils\translation\trans_real.py", line 140, in _new_gnu_trans
fallback=use_null_fallback)
File "C:\Python27\Lib\gettext.py", line 481, in translation
t = _translations.setdefault(key, class_(fp))
File "C:\Python27\Lib\gettext.py", line 182, in __init__
self._parse(fp)
File "C:\Python27\Lib\gettext.py", line 318, in _parse
self.plural = c2py(plural)
File "C:\Python27\Lib\gettext.py", line 95, in c2py
raise ValueError, 'plural forms expression could be dangerous'
ValueError: plural forms expression could be dangerous
I've read about that (plural forms expression could be dangerous and Django: gettext raises ValueError: 'plural forms expression could be dangerous'), but the only solution I found is to remove lines referred to plural translations, for example:
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
If I remove those lines in my translations files, it still raises the same error, because it takes into account other files located at the virtual env.
Update 2: If I run the server from the command line, using
python manage.py runserver
there's no problem, everything is going right. However, when I run the project from Visual Studio, the error is raised.
Is there any way to solve this problem?
Thanks in advance.
I want build a blog with Python 2.7 and Django 1.7.8.
When I use Django,I keep getting an error: UnicodeDecodeError.
The relevant code is:
#coding: utf-8
from django.db import models
class Entry(models.Model):
title = models.CharField(max_length=200)
body = models.TextField()
publish = models.BooleanField(default=False)
And ./manage.py makemigrations get an error:
Migrations for 'blog':
0001_initial.py:
- Create model Entry
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/core/management/commands/makemigrations.py", line 124, in handle
self.write_migration_files(changes)
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/core/management/commands/makemigrations.py", line 143, in write_migration_files
migrations_directory = os.path.dirname(writer.path)
File "/usr/local/lib/python2.7/dist-packages/Django-1.7.8-py2.7.egg/django/db/migrations/writer.py", line 222, in path
return os.path.join(basedir, self.filename)
File "/usr/lib/python2.7/posixpath.py", line 80, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 10: ordinal not in range(128)
I don't see what's going wrong, any idea? I already add coding:utf-8 in top of code
do you have data in the models already?
if so, i am guessing you have some bad characters in there.
do you have a unicode or str method defined on the models?
if so, i suggest using them in a try statement of sorts like this:
def __str__():
try:
return "%s" % self.title
except:
return "%s" % self.pk
when you see only the PK, you will know that your 'title' field has bad data.
extend this to include whichever fields you want to display, not just title.
it looks like it fails when joining the path?
maybe you have folder names that can't be converted to ascii?
also , consider editing the file that is having the error? maybe add a print statement just before the line that fails that shows what is trying to be joined?
edit the file mentioned below, add the print statement to see what is trying to be joined?
File "/usr/lib/python2.7/posixpath.py", line 80, in join
path += '/' + b
I just generated the migration scripts through ./manage.py schemamigration --auto and ran it. I get the following error. I am stumped as to what it could mean. I have been using SET_NULL for a while now. So this is something new that didn't occur earlier. Any idea what could be wrong?
Traceback (most recent call last):
File "./manage.py", line 16, in
execute_from_command_line(sys.argv)
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/django/core/management/init.py", line 399, in execute_from_command_line
utility.execute()
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/django/core/management/init.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.dict)
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/south/management/commands/schemamigration.py", line 111, in handle
old_orm = last_migration.orm(),
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/south/utils/init.py", line 62, in method
value = function(self)
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/south/migration/base.py", line 432, in orm
return FakeORM(self.migration_class(), self.app_label())
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/south/orm.py", line 48, in FakeORM
_orm_cache[args] = _FakeORM(*args)
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/south/orm.py", line 134, in init
self.retry_failed_fields()
File "/home/vivekv/.environments/fantain/local/lib/python2.7/site-packages/south/orm.py", line 377, in retry_failed_fields
fname, modelname, e
ValueError: Cannot successfully create field 'winner' for model 'match': 'module' object has no attribute 'SET_NULL'.
OK This is not a valid question. I am embarrassed to admit I made a small tweak on the migration script that caused the problem. Please ignore this question - seems like I dont have a way to delete a question I had asked!
I wrote the solution here
Short answer: use models.SET_NULL instead of "SET_NULL".
You need to replace this in all migrations which have b"SET_NULL"