I could see a couple of examples to read from the http stream. But how to write to a http input stream using python?
You could use standard library module httplib: in the HTTPConnection.request method, the body argument (since Python 2.6) can be an open file object (better be a "pretty real" file, since, as the docs say, "this file object should support fileno() and read() methods"; but it could be a named or unnamed pipe to which a separate process can be writing). The advantage is however dubious, since (again per the docs) "The header Content-Length is automatically set to the correct value" -- which, since headers come before body, and the file's content length can't be known until the file is read, implies the whole file's going to be read into memory anyway.
If you're desperate to "stream" dynamically generated content into an HTTP POST (rather than preparing it all beforehand and then posting), you need a server supporting HTTP's "chunked transfer encoding": this SO question's accepted answer mentions that the popular asynchronous networking Python package twisted does, and gives some useful pointers.
Related
I am having a hard time with this. Is there a way to get a compiled protocol buffer file’s (pb2.py) contents into excel?
Your question lacks detail and a demonstration of your attempt to solve this problem, it is likely that it will be closed.
Presumably (!?) your intent is to start with serialized binary/wire format protocol buffer messages, unmarshal these into Python objects and then, using a Python package (list that can interact with Excel, enter these objects as row into Excel.
The Python (pb2.pby) file generated by the protocol buffer compiler (protoc) from a .proto file, contains everything you need to marshal and unmarshal messages in the binary/wire format to Python objects that represent the messages etc. that are defined by the .proto file. The protocol buffer documentation is comprehensive and explains this well (link).
Once you've unmarshaled the data into one or more Python objects, you will need to use the Python package for Excel of your choosing to output these objects into the spreadsheet(s).
It is unclear whether you have flat or hierarchical data. If you have anything non-trivial, you'll also need to decide how to represent the structure in the spreadsheet's table-oriented structure.
I've searched all over the internet and the "safest" way to check uploaded file type seems to be checking magic numbers/headers using python-magic library
magic.from_buffer(open("testdata/test.pdf").read(2048)) for in memory files
magic.from_file("testdata/test.pdf", mime=True) for files on disk
However what I also read everywhere that even magic bytes can be spoofed with a security consequences.
Is there a safer way or at least another-layer-safe solution in python?
(Preferrably, but not limited to, using Django)
PS: Stuff like "checking an extension" or "Content-type" header from user side - I don't consider a file type check at all.
I hava a code in lua produce images and other data containers then these data should be used in another python script.
i tried to use (os.execute(command)) as it shown here https://www.lua.org/pil/22.2.html but i don't know how to pass these parameters direct from lua to python scriprt as arguments
Command-line arguments can only be strings—and only relatively short strings, and only strings which can be handled by your default system encoding.
If you need to pass data that's large, or structured, or just arbitrary binary bytes, you can't use command-line arguments.
The simplest solution is to save the data in a file, then pass the filename as a command-line argument. Then, on the Python side, you just open and read the same file.
You can also stream data into the child process's standard input. Lua doesn't come with anything like Python's subprocess module, but sometimes popen is sufficient.
When popen isn't sufficient, you can build it yourself atop POSIX or Windows API calls. See this answer for an example of doing it for POSIX.
You can also just open a pipe, shared memory segment, etc. for the child to inherit, and then pass a file descriptor or key as a command-line argument.
Finally, you can write a simple socket server (possibly using a higher-level protocol that libraries already exist for ranging from Netstrings over UDP up to HTTP) and pass the port number as a command-line argument.
The question is about API design. The scenario is, I have some bytes of KNOWN length in memory and I want to design a Pythonic API that flushes them to AWS S3 with few RPCs. By Pythonic, I mean if there is already an accepted API doing this, I want to copy it.
One way is to do something like io.create(filename, content). This can be translated into a single HTTP request using S3 XML API.
The traditional open, write, and close pattern is a streaming API. open doesn't accept a length argument so the created stream doesn't know it should buffer all writes into a single RPC.
Of course the API can look like:
with open(filename, buffering="UNLIMITED") as f:
f.write(content)
But buffering doesn't really support "UNLIMITED" constant.
So what can I do? File a PEP? Thanks.
NFS, at least in version 2 and 3, isn't terribly good at doing locking or atomicity.
The best bet, again at least in NFS 2 and 3, is to create your file with a temporary name, and then use os.rename().
Perhaps some progress has been made on this in NFS 4; I don't know.
I have a proto file defining some GPB (proto buffer) messages.
I want to implement a simple python script that go over the different messages and write to external file (lets say a JSON file) the basic information regarding each of the messages' fields (name, type, default value, etc..).
I searched on the WEB and found that once I get the GPB descriptor the rest should be relatively easy.
However, I have no idea how to get the descriptor itself.
Can someone help me here??
10x
protoc has an option --descriptor_set_out which writes the descriptors as a FileDescriptorSet as described in descriptor.proto from the Protobuf source code. See protoc --help for more info.
Alternatively, you might consider actually writing your script as a code generator plugin. In this case, you wouldn't be generating code, but just a JSON file (or whatever), but the mechanism is the same.