I am trying to create a system that enables the admin to upload a zipfile, then the script will automatically, using signals, unzip it, search for all the files in jpg,png. create a list of them and generate a database record according to it.
In models, i have Project and Photo table, Photo has Many-to-One aka Foreign Key relationship with Project.
The script below is with the signal i am working. I can get instance.file_zip.path without errors, and the script works well when run manually.
Long-time debugging and I assume that there is something wrong with belongs_to=instance but I do not know how to fix it as I didn't actually understand why it gaves an error.
The extraction part works fine, I just put them here for reference, most likely you will not need to read and understand it.
#receiver(post_save, sender=Project)
def unzip_and_process(sender, instance, **kwargs):
#project_zip = FieldFile.open(file_zip, mode='rb')
file_path = instance.file_zip.path
file_list = []
with zipfile.ZipFile(file_path, 'r') as project_zip:
project_zip.extractall(re.search('[^\s]+(?=\.zip)', file_path).group(0))
project_zip.close()
for root, dirs, files in os.walk(file_path):
for filename in files:
file_list.append(os.path.join(root, filename))
photo_list = filter(filter_photos, file_list)
for photo in photo_list:
print 'Processing %s'%photo
p = Photo.objects.create(belongs_to=instance, img=photo, desc='Processed from zipfile')
p.save()
update
class Project(models.Model):
project_name=models.CharField(max_length=150)
date_taken=models.DateField()
date_deadline=models.DateField()
price=models.FloatField()
price_paid=models.BooleanField()
owner=models.ForeignKey(User)
file_zip=models.FileField(upload_to='projects/%Y/%m/%d')
def __unicode__(self):
return self.project_name
def file_path(self):
return re.search('[^\s]+(?=\.zip)', self.file_zip.name).group(0)
class Photo(models.Model):
def project_path(instance, filename):
return '%s/%s'%(instance.belongs_to.file_path(),filename)
belongs_to=models.ForeignKey(Project, verbose_name="related_project")
img=models.ImageField(upload_to=project_path, max_length=255)
desc=models.CharField(max_length=255)
def __unicode__(self):
return '%s FROM [%s]'%(self.img.name,self.belongs_to)
django-photologue has something extractly what you want, and they created a similiar hack to upload zipfile.
Link: http://code.google.com/p/django-photologue/ incase you don't want to google
Evenmore, the zip uploading class is GalleryUpload(models.Model)
for root, dirs, files in os.walk(file_path):
file_path refers to a zip file. not a directory hence os.walk returns nothing
Related
I've a small problem, I have a method in another class that scans a folder and for each file in that folder I need to create new folders according to the date of the files to organize and initiate a new instance, but I'm having trouble on the second part because whatever I try to return it dont seems to work since no new instances are created.
class CPImages:
def __init__(self, filename):
self.filename = filename
self.exif = {}
self.metadata = {}
#staticmethod
def makeCPImage(filename):
image = CPImages(filename)
image.loadExif()
date = photo.getDate()
if not (os.path.exists(date)):
#if the folder dont exist, create one and then copy the file
os.makedirs(date)
CPImages.copyToFolder(filename, date)
else:
#if the folder exists just copy
CPImages.copyToFolder(filename, date)
return CPImages(filename)
Just a bit of more context, the method loadExif() extracts the Exif of the image, the method getDate() turns the date into an usefull format.
Here you have the code for searching in the folders
class ImageCollection:
#Here I search in all folders to extract all files
def allFiles(folder, fileslist=[], extension='*.jpg*'):
for obj in os.scandir(folder):
if obj.is_dir():
ImageCollection.allFiles(obj)
if obj.is_file:
if fnmatch.fnmatch(obj, extension):
fileslist.append(os.path.join(obj))
else:
pass
return fileslist
#Here I use the files to pass to makeCPImages
def scanFolder(folder):
for i in ImageCollection.allFiles(folder):
CPImages.makeCPImage(i)
ImageCollection.scanFolder('somePath')
Use #classmethod, instead of #staticmethod.
you can refer to this answer. for further info.
I'm working on a news blog where you can add as many files to news as you want. For files storing properties, I'm using amazon s3 and django-strorage. But after I've added news-update view, I got some problems with files management.
As you can see, here my files model
class FileStorage(models.Model):
file = models.FileField(upload_to=uploadFile)
upload_path = models.TextField(blank=True, default='files/')
def __str__(self):
return f'Файл: {self.file.name.split("/")[-1]}'
The main problem is how to update FileField after moving the file into another directory?
Here is my files moving script.
bucket = S3Boto3Storage()
from_path = bucket._normalize_name(bucket._clean_name(self.instance.file.name))
to_path = bucket._normalize_name(bucket._clean_name(self.cleaned_data['upload_path']))
result = bucket.connection.meta.client.copy_object(
Bucket=bucket.bucket_name,
CopySource=bucket.bucket_name + "/" + from_path,
Key=to_path)
bucket.delete(from_path)
All works good, but only with path.
File in FileField store old path.
How can I update it to?
Screen with problem
If you want this, just change your file "name" parameter like this:
file.name = "new file path"
I want to create random uid file names for images uploaded with the django-ckeditor/uploader.
I've created utils.py in the same folder as settings.py:
import uuid
def get_name_uid():
ext = filename.split('.')[-1]
filename = "%s.%s" % (uuid.uuid4(), ext)
return filename
I would like to add this "random" file name to settings.py:
CKEDITOR_FILENAME_GENERATOR = get_name_uid()
How can I do this? I am not sure how to get the filename that is uploaded in the editor. Should I pass the filename from settings.py to utils.py? Or is there a different way to do this?
Their documentation says the following:
``CKEDITOR_UPLOAD_PATH = "uploads/"``
When using default file system storage, images will be uploaded to "uploads" folder in your MEDIA_ROOT and urls will be created against MEDIA_URL (/media/uploads/image.jpg).
If you want be able for have control for filename generation, you have to add into settings yours custom filename generator.
```
# utils.py
def get_filename(filename):
return filename.upper()
```
```
# settings.py
CKEDITOR_FILENAME_GENERATOR = 'utils.get_filename'
```
CKEditor has been tested with django FileSystemStorage and S3BotoStorage.
There are issues using S3Storage from django-storages.
It's basically all spelled out for you in the docs:
def get_filename(filename):
return filename.upper() # returns the uppercase version of filename
So the example function get_filename gets the uploaded filename passed in and you're supposed to return the filename you want it to be. This is what we call a callback.
What the callback gets passed in as arguments is called the "callback signature" and the docs specify neatly what it gets.
So put the function in a place where it makes sense. I would choose mysite/mysite/utils.py given the structure outlined in the tutorial, under the heading "Let’s look at what startproject created:". So in the same directory as settings.py. I would name it generate_uuid4_filename and mysite/mysite/utils.py would look like this:
import uuid
import os.path
def generate_uuid4_filename(filename):
"""
Generates a uuid4 (random) filename, keeping file extension
:param filename: Filename passed in. In the general case, this will
be provided by django-ckeditor's uploader.
:return: Randomized filename in urn format.
:rtype: str
"""
discard, ext = os.path.splitext(filename)
basename = uuid.uuid4().urn
return ''.join(basename, ext)
Now update your settings.py:
# Near the rest of the CKEditor variables
CKEDITOR_FILENAME_GENERATOR = '<app_label>.utils.generate_uuid4_filename'
And you're done. Good luck!
I want to save downloaded images to a path that like so: media/images/[user.pk]/ directly from urlretrieve from urllib library is that possible and what is the most elegant way?
This are the used code snippets:
here I use urlretrieve with path like .../media/images/[user.pk]/[filename.jpg]
file_path, header = urlretrieve(image_url, os.path.join(
MEDIA_ROOT, get_image_path(
self, 'profile_image_'+str(
datetime.utcnow().strftime("%Y%m%d_%H%M"))+'.jpg
Here is the defined function that returns desired path with filename
def get_image_path(instance, filename):
return os.path.join('images', str(instance.pk), filename)
when I run I get an error since the file does not exists:
FileNotFoundError at /rest/oauth2/google
[Errno 2] No such file or directory: 'C:\xampp\htdocs\BookProjectFresh\media\images\21\profile_image_20160723_0801.jpg'
I know I can achieve that by first loading file into temp folder and then load from there and put into the django model and it will automatically create file path if it does not exists, but then I have 2 files on my pc with same content. Or would be the best way to do it with os.path.makedirs? Please if you know any other way share it.
I solved a problem like so:
def save_image_from_url(self, image_url):
if self.profile_image_url != image_url or not os.path.isfile(self.profile_image.path):
if not os.path.exists(get_image_path(self, os.path.join(MEDIA_ROOT, get_image_path(
self, '')))):
os.makedirs(get_image_path(self, os.path.join(MEDIA_ROOT, get_image_path(
self, ''))))
self.profile_image_url = image_url
file_path, header = urlretrieve(image_url, os.path.join(MEDIA_ROOT, get_image_path(
self, 'profile_image_'+str(datetime.utcnow().strftime("%Y%m%d_%H%M"))+'.jpg')))
self.profile_image = file_path
self.save()
In my view callable, I want users to be able to create a new file called filename like so:
#view_config(route_name='home_page', renderer='templates/edit.pt')
def home_page(request):
if 'form.submitted' in request.params:
name= request.params['name']
input_file=request.POST['stl'].filename
vertices, normals = [],[]
for line in input_file:
parts = line.split()
if parts[0] == 'vertex':
vertices.append(map(float, parts[1:4]))
elif parts[0] == 'facet':
normals.append(map(float, parts[2:5]))
ordering=[]
N=len(normals)
...parsing data...
data=[vertices,ordering]
jsdata=json.dumps(data)
renderer_dict = dict(name=name,data=jsdata)
app_dir = request.registry.settings['upload_dir']
filename = "%s/%s" % ( app_dir , name )
html_string = render('tutorial:templates/view.pt', renderer_dict, request=request)
with open(filename,'w') as file:
file.write(new_comment)
return HTTPFound(location=request.static_url('tutorial:pages/%(pagename)s.html' % {'pagename': name}))
return {}
right now, when I attempt to upload a file, I am getting this error message: IOError: [Errno 2] No such file or directory: u'/path/pages/one' (one is the name variable) I believe this is because I am incorrectly defining the app_dir variable. I want filename to be the url of the new file that is being created with the name variable that is defined above (so that it can be accessed at www.domain.com/pages/name). Here is the file structure of my app:
env
tutorial
tutorial
templates
home.pt
static
pages
(name1)
(name2)
(name3)
....
views.py
__init__.py
In my init.py I have:
config.add_static_view(name='path/pages/', path=config.registry.settings['upload_dir'])
In my development.ini file I have
[app:main]
use = egg:tutorial
upload_dir = /path/pages
Edit: If anyone has an idea on why this question isn't getting much attention, I would love to hear it.
While I feel like you probably have a misunderstanding of how to serve up user-generated content, I will show you a way to do what you're asking. Generally user-generated content would not be uploaded into your source, you'll provide some configurable spot outside to place it, as I show below.
Make the path configurable via your INI file:
[app:main]
use = egg:tutorial
upload_dir = /path/to/writable/upload/directory
Add a static view that can serve up files under that directory.
config.add_static_view(name='/url/to/user_uploads', path=config.registry.settings['upload_dir'])
In your upload view you can get your app_dir via
app_dir = request.registry.settings['upload_dir']
Copy the data there, and from then on it'll be available at /url/to/user_uploads/filename.