I have a view in my Django application that automatically creates an image using the PIL, stores it in the Nginx media server, and returns a html template with a img tag pointing to it's url.
This works fine, but I notice an issue. For every 5 times I access this view, in 1 of them the image doesn't render.
I did some investigation and I found something interesting, this is the HTTP response header when the image renders properly:
Accept-Ranges:bytes
Connection:keep-alive
Content-Length:14966
Content-Type:image/jpeg
Date:Wed, 18 Aug 2010 15:36:16 GMT
Last-Modified:Wed, 18 Aug 2010 15:36:16 GMT
Server:nginx/0.5.33
and this is the header when the image doesn't load:
Accept-Ranges:bytes
Connection:keep-alive
Content-Length:0
Content-Type:image/jpeg
Date:Wed, 18 Aug 2010 15:37:47 GMT
Last-Modified:Wed, 18 Aug 2010 15:37:46 GMT
Server:nginx/0.5.33
Notice the Content-Lenth equals to zero. What could have caused this? Any ideas on how could I further debug this problem?
Edit:
When the view is called, it calls this "draw" method of the model. This is basically what it does (I removed the bulk of the code for clarity):
def draw(self):
# Open/Creates a file
if not self.image:
(fd, self.image) = tempfile.mkstemp(dir=settings.IMAGE_PATH, suffix=".jpeg")
fd2 = os.fdopen(fd, "wb")
else:
fd2 = open(os.path.join(settings.SITE_ROOT, self.image), "wb")
# Creates a PIL Image
im = Image.new(mode, (width, height))
# Do some drawing
.....
# Saves
im = im.resize((self.get_size_site(self.width),
self.get_size_site(self.height)))
im.save(fd2, "JPEG")
fd2.close()
Edit2: This is website:
http://xxxcnn7979.hospedagemdesites.ws:8000/cartao/99/
if you keep hitting F5 the image on the right will eventually render.
We had this problem a while back when writing HTML pages out to disk. The solution for us was to write to a temporary file and then atomically rename the file. You might also want to consider using fsync.
The full source is available here: staticgenerator/__init__.py, but here are the useful bits:
import os
import stat
import tempfile
...
f, tmpname = tempfile.mkstemp(dir=directory)
os.write(f, content)
# See http://docs.python.org/library/os.html#os.fsync
f.flush()
os.fsync(f.fileno())
os.close(f)
# Ensure it is webserver readable
os.chmod(tmpname, stat.S_IREAD | stat.S_IWRITE | stat.S_IWUSR | stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP | stat.S_IROTH)
# Rename is an atomic operation in POSIX
# See: http://docs.python.org/library/os.html#os.rename
os.rename(tmpname, fn)
Related
I'm trying to insert an image into a libreoffice document that is handled/controlled by unotools.
Therefore I start LibreOffice with this command:
soffice --accept='socket,host=localhost,port=8100;urp;StarOffice.Service'
Inside my python code I can connect to LibreOffice:
from unotools import Socket, connect
from unotools.component.writer import Writer
context = connect(Socket('localhost', 8100))
writer = Writer(context)
(This code is taken from this documentation: https://pypi.org/project/unotools/)
By using writer.set_string_to_end() I can add some text to the document. But I also want to insert an image into the document. So far I couldn't find any resource where this was done. The image is inside of my clipboard, so ideally I want to insert the image directly from there. Alternatively I can save the image temporarily and insert the saved file.
Is there any known way how to insert images by using unotools? Any alternative solution would also be great.
I've found a way to insert images by using uno instead of unotools:
import uno
from com.sun.star.awt import Size
from pythonscript import ScriptContext
def connect_to_office():
if not 'XSCRIPTCONTEXT' in globals():
localContext = uno.getComponentContext()
resolver = localContext.ServiceManager.createInstanceWithContext(
'com.sun.star.bridge.UnoUrlResolver', localContext )
client = resolver.resolve("uno:socket,host=localhost,port=8100;urp;StarOffice.ComponentContext" )
global XSCRIPTCONTEXT
XSCRIPTCONTEXT = ScriptContext(client, None, None)
def insert_image(doc):
size = Size()
path = uno.systemPathToFileUrl('/somepath/image.png')
draw_page = self.doc.DrawPage
image = doc.createInstance( 'com.sun.star.drawing.GraphicObjectShape')
image.GraphicURL = path
draw_page.add(image)
size.Width = 7500
size.Height = 5000
image.setSize(size)
image.setPropertyValue('AnchorType', 'AT_FRAME')
connect_to_office()
doc = XSCRIPTCONTEXT.getDocument()
insert_image(doc)
sources:
https://ask.libreoffice.org/en/question/38844/how-do-i-run-python-macro-from-the-command-line/
https://forum.openoffice.org/en/forum/viewtopic.php?f=45&t=80302
I still don't know how to insert an image from my clipboard, I worked around that problem by saving the image first. If someone knows a way to insert the image directly from the clipboard that would still be helpful.
Can someone tell me how to save a file in redis?
I'd like to be able to do something like:
hmset 12345 sound_bytes <pathtofile/filename.mp3>
as an example.
I'm using python and so will need to open the mp3 file using python, save it as a byte array I guess (?) and then save that byte array into redis.
I know I can open a file like so in python:
with open(filename, 'rb') as fd:
contents = fd.read()
But if i were to somehow save "contents" in REDIS, when I retrieve it later on to actually play it or pass it back via a REST method, is there anything else that needs doing? What do I need to do to have python "recognize" this as a sound file and not just a string?
I've never tried something like this before, so any pointers or gotchas would be appreciated.
EDIT 1
So far, this is the code I've been playing around with:
14 def sound_file_to_bytes(pathtofile):
15 try:
16 with open(pathtofile, 'rb') as fd:
17 contents = fd.read()
18 logging.info(contents)
19 fd.close()
20 return contents
21 except Exception as ex:
22 return "Error:", ex
23
24 def sound_as_bytes_to_db(soundbytes):
25 try:
26 logging.info('attempting to save sound bytes')
27 my_redis = redis.Redis(connection_pool=POOL)
28 response = my_redis.hmset('55555', 'greeting', soundbytes)
29 logging.info(response)
30 return True
31 except Exception as ex:
32 return False
Through the log, I can see that the contents of the file is being read into my variable. I don't get any errors when I try to turn around and write to the database, but the contents of the value of the 'greeting' key in my SET is empty.
Notice the redis output below:
127.0.0.1:6379[5]> hgetall 55555
1) "email1"
2) "johndoe#hotmail.com"
3) "email2"
4) "jd#yahoo.com"
5) "greeting"
6) ""
127.0.0.1:6379[5]>
EDIT 2
I found out why it wasn't saving the contents to the database. I had syntax problem with the hmset command. now the code looks like this:
28 response = my_redis.hmset('55555',{'greeting':soundbytes})
I have to write xml data to file and if file exists I have to remove that file first but it appends the data every time to the same file.
Note: I am doing this in scons file
myfile.py
class AttributeDataXmlBuilder():
def __init__(self, AttrXml, simAttrXml, logger = logging.getLogger(__name__)):
self.logger = logger
self.simAttrXml = os.path.join(simAttrXml, "AttributeData.xml")
self.AttrXml = AttrXml
assert len(AttrXml) > 0, 'Need at least two xml file to merge'
self.roots = [ElementTree.parse(xml_file).getroot() for xml_file in self.AttrXml]
def combine(self):
# iteratively add all files to first file
for root_tag in self.roots[1:]:
self.combine_element(self.roots[0], root_tag)
# iterate final file for readaccess and writeaccess tag and change there value to NoAccess
for read_access in self.roots[0].iter('ReadAccess'):
if read_access.text != 'NoAccess':
read_access.text = 'NoAccess'
for write_access in self.roots[0].iter('WriteAccess'):
if write_access.text != 'NoAccess':
write_access.text = 'NoAccess'
# set owner as System
self.roots[0].attrib['owner'] = 'System'
ATT_DATA = ElementTree.ElementTree(self.roots[0])
ATT_DATA.write(self.simAttrXml, encoding="utf-8", xml_declaration=True)
def combine_element(self, one, other):
one.extend(other)
SConscript
myfile.XmlBuilder(DataXml, sDataXml_loc).combine()
From the Python documentation on open:
mode is an optional string that specifies the mode in which the file is opened. It defaults to 'r' which means open for reading in text mode. Other common values are 'w' for writing (truncating the file if it already exists) ...
In plain English, this means that when you open the file for writing its existing contents are immediately deleted and whatever you write to the file (if anything) becomes its new content. You say
"if file exists I have to remove that file first"
so it looks as though the ElementTree's write method is doing something weird. I presume you have checked that the file size increases after each run, rather than a simple visual inspection of the output? I'd be very surprised if ElementTree outputs were so different from standard Python files.
EDIT: The following program creates an XML file, parses it and writes a subtree out to a new file:
from xml.etree.ElementTree import ElementTree
data = """\
<html>
<head>
<title>Example page</title>
</head>
<body>
<p>Moved to example.org
or example.com.</p>
</body>
</html>
"""
with open('input.xml', 'w') as f:
f.write(data)
tree = ElementTree()
tree.parse('input.xml')
p = tree.find("body/p") # Finds first occurrence of tag p in body
links = list(p.iter("a")) # Returns list of all links
for i in links: # Iterates through all found links
i.attrib["target"] = "blank"
tree.write("output.xhtml")
I run it once:
(static_web) Airhead:tmp sholden$ ls -l input.xml output.xhtml
-rw-r--r-- 1 sholden wheel 225 15 Mar 10:23 input.xml
-rw-r--r-- 1 sholden wheel 254 15 Mar 10:23 output.xhtml
I run it again:
(static_web) Airhead:tmp sholden$ ls -l input.xml output.xhtml
-rw-r--r-- 1 sholden wheel 225 15 Mar 10:23 input.xml
-rw-r--r-- 1 sholden wheel 254 15 Mar 10:23 output.xhtml
and the files are the same length. Not even the creation time has changed, since they have been truncated rather than removed and rewritten. You could add an os.remove call to verify it made no difference to the file sizes.
These are two citygml files:
an example file from the citygml website with some changes done to the file manually (https://drive.google.com/file/d/0Bx7bUIyq-ZEWY3FRb1hTbEU0SGc/view?usp=sharing)
one generated by my python & lxml code in an attempt to recreate the example file (https://drive.google.com/file/d/0Bx7bUIyq-ZEWaU1tNU1Wc01XNW8/view?usp=sharing)
When i open the 2 files with notepad ++ and go through them line by line, the python generated file is identical to the example file, however when I try to validate the python-generated file with the citygml xsd schema it failed, while the example file passed. Why is this the case, since they are identical, am I doing something wrong ? (I am new to dealing with xml files)
This is a snippet of my python code, the xml file built using etree (since I cant post more than two links for you to download my code):
et = ElementTree(root) cityObjectMember = SubElement(root,'cityObjectMember') luse = SubElement(cityObjectMember, "{" + XMLNamespaces.luse+ "}" +'Landuse') luse.attrib["{" + XMLNamespaces.gml+ "}" +'id'] = 'ID_ground_texture'
Thanks in advance.
Don't trust your eyes, let the computer compare the files:
$ diff -u0 landusetest_validated.gml landusetest_cnntbevalidated.gml
--- landusetest_validated.gml 2015-12-28 14:33:34.437635671 +0100
+++ landusetest_cnntbevalidated.gml 2015-12-28 14:33:34.437635671 +0100
## -4,2 +4,2 ##
- <luse:LandUse gml:id="ID_ground_texture">
- <gml:name>Simple textured patch</gml:name>
+ <luse:Landuse gml:id="ID_ground_texture">
+ <gml:name>simple textured patch</gml:name>
## -26 +26 ##
- </luse:LandUse>
+ </luse:Landuse>
Case does matter. Landuse != LandUse.
The follwowing is a code written in blender python
import bpy
import os
import subprocess
import time
bpy.data.materials["Material"].use_face_texture = True
customer_id = "john33"
image1 = "orangeskin.jpg" # define image 1 from folder
imagepath1 = "C:\\Users\\mrryan\\Desktop\\models\\materialsjpeg\\"
image2 = "elec.jpg"
imagepath2 = "C:\\Users\\mrryan\\Desktop\\models\\materialsjpeg\\"
#Step 1 go to Edit Mode
bpy.ops.object.editmode_toggle()
#step 2 insert image in the Image Editor
bpy.context.area.type = 'IMAGE_EDITOR'
bpy.ops.uv.smart_project()
bpy.ops.image.open(filepath = image1, directory= imagepath1 ,
files=[{"name":image1, "name":image1}])
#Step 3 back to Object Mode TRY AND CHANGE TITLE TO `CUSTOMER_ID AND FILE`
bpy.ops.object.editmode_toggle()
bpy.data.window_managers["WinMan"].sketchfab.title = "orangetube"
#Step 4 save new.blend
filepath = os.path.join(r"C:\Users\mrryan\Desktop", customer_id + "orangytube.blend")
bpy.ops.wm.save_as_mainfile(filepath = filepath)
toggleedit = bpy.ops.object.editmode_toggle()
#here is the problem!!!!
subprocess.call(["bpy.ops.export.sketchfab()"], shell = True)
#new texture
#Step 1 go to Edit Mode
#step 2 insert image in the Image Editor
bpy.context.area.type = 'IMAGE_EDITOR'
bpy.ops.image.open(filepath= image2, directory= imagepath2,
files= [{"name":image2,"name":image2}])
bpy.ops.uv.smart_project()
#Step 3 back to Object Mode
bpy.ops.object.editmode_toggle()
bpy.data.window_managers["WinMan"].sketchfab.title = "elec-tube"
#Step 4 save new.blend
filepath = os.path.join(r"C:\Users\mrryan\Desktop", customer_id + "elec-tube.blend")
bpy.ops.wm.save_as_mainfile(filepath = filepath)
bpy.ops.export.sketchfab()
The idea is, eventually to programmatically change the active object( which is combined but not joined with another object) to have different materials and change the shape, etc. Each individual model combination would be uploaded to sketchfab separately and saved as a blender file.
At this stage my problem is that, while uploading, the script does not wait until the first upload is finished.This generates an error that says to please wait for the current upload to finish. and as a result only one model is uploaded.
subprocess.Popen
has been tried but i could not get it to work.
Perhaps the return values were wrong?
How can you get the script to upload the first model, wait until finished, then continue to make the adjustments and upload the second?
(I work at Sketchfab, but I don't have much experience in Blender scripting).
The issue is that the Sketchfab Blender Exporter uses a thread for the upload, meaning the subprocess will return, with a thread still running in the background.
I would experiment in launching bpy.ops.export.sketchfab() using a new Thread and join(), instead of just subprocess.
The following resources might help:
python multithreading wait till all threads finished
https://docs.python.org/2/library/multiprocessing.html
http://www.blender.org/documentation/blender_python_api_2_62_2/info_gotcha.html#strange-errors-using-threading-module
Let us know how it goes!