I don't really know if this even classifies as a problem, but it somehow surprised me, since I thought that Docker containers run in sandboxed enviroments.
I have a running docker-container that includes a django-rest server.
When making the appropiate requests from the frontend (that's also served through a seperate NGINX-container in the same docker network) the request successfully triggers the response of the django server that includes writing the contents of the requests to a text-file.
Now, the path to the text file points to a subdirectory in the folder where the script itself is located.
But now the problem: the file is saved in the directory on the host-machine itself and (to my knowledge) not in the container.
Now is that something that docker is capable of doing or am I just not aware that django itself (which is running in the WSGI development server mind you) writes the file to the host machine.
now = datetime.now()
timestamp = datetime.timestamp(now)
polygon = json.loads(data['polygon'])
string = ''
for key in polygon:
string += f'{key}={polygon[key]}\r\n'
with open(f"./path/to/images/image_{timestamp}.txt", "w+") as f:
f.write(string)
return 200
Is there something I am missing or something I do not understand about docker or django yet?
Related
I'm trying to upload a .mp4 video to be processed by Python server in Heroku. In my server.py, I've have following code:
# Initialize the Flask application
MYDIR = os.path.dirname(__file__)
app = Flask(__name__)
app.config['UPLOAD_PATH'] = 'uploads'
cors = CORS(app, resources={r"/api/*": {"origins": "*"}})
# route http posts to this method
#app.route('/api/uploadFile', methods=['POST'])
def uploadFile():
r = request # this is the incoming request
uploaded_file = r.files['uploadedfile']
# save the incoming video file locally
if uploaded_file.filename != '':
content = r.files['uploadedfile'].read()
uploaded_file.save(os.path.join(MYDIR, uploaded_file.filename))
However, when I run heroku run bash and do ls I don't see the file at all. I've looked at the previous posts regarding this and doing what many have suggested, i.e., to have an absolute path os.path.join(MYDIR, uploaded_file.filename). But I still don't see the file. I know that Heroku is an ephemeral system and doesn't store files, but all I want to do is process the video file and after it's done, I'll myself delete it. Oh by the way, the video is approx 25MB in size. I tried a low quality version which is around 4MB and the problem still persists, so my guess is it is not the size of the video that's the problem.
Also, when I read the content via content = r.files['uploadedfile'].read(), I do see that content is approximately of length 25M suggesting that I'm indeed receiving the video correctly.
Ideally, I would love to convert the byte stream to something like list of frames and then process it using opencv. But I thought, I can, as a first step, just save it locally, then read and process. I don't care that it is deleted after 5 minutes of my processing is done.
Any help would be greatly appreciated. Banged my head around for a day looking for a solution. Thanks!!!
Just want to add that the commands above give me no error. So it seems like everything worked but then when I do heroku run bash, I see no file anywhere. It would've been great if there was an error or an warning provided by the system :-(.
Also, I tried running things locally on localhost server and it does store the file. So it works locally on my machine, just can't seem to save it in Heroku.
I'm currently dealing with a problem in my django web application that is running locally on 192.168.0.168. I'm making use of an OCR library that makes use of .wasm.gz files and .data files. In the documentation they mention: " In some cases you need to configure your webserver to serve data and wasm.gz files.". My first idea was to add the following mimetypes in django:
mimetypes.add_type("application/x-gzip", ".wasm.gz", True)
mimetypes.add_type("application/octet-stream", ".data", True)
I'm however still seeing the error: error wasm-instantie. It somehow can't instantiate wasm because django needs to serve the wasm.gz and .data files correctly. These have been added however to the static folder and I'm able to download them if I manually browse to the link. Does anyone have any idea what I need to do to serve these correctly?
If the library you're using loads WASM code with WebAssembly.instantiateStreaming, the WASM files must be served with the application/wasm content type.
GZip is a content encoding rather than the media type.
I'm assuming you're using the dev server, so if the above is the problem then you should replace the first line of code above with:
mimetypes.add_type("application/wasm", ".wasm.gz", True)
(Note that the above is only necessary when running Python versions pre 3.8. If you're running Python 3.8 or later then this is not your solution.)
To clarify, it's fairly common practice to pre-compress static assets with GZip, which are then sent over HTTP with the Content-Encoding: gzip header set. This saves server resources compressing the file every time it's requested.
When set up properly, the browser receives the GZipped bytes, decompresses them, and interprets the decompressed bytes as the appropriate Content-Type.
The Django dev server defers to the built in mimetypes package, which does this testing.
I am hosting an Angular project on a web server. On the server there are python scripts that are scheduled to run and generate a .json file. This .json saves over the existing .json in the projects assets file (dist/test/assets).
When this occurs the .json is replaced but the data on the web page does not get updated.
Is there a way to get the project to update the data using a process similar to this?
Alternately, is there a way to pull the data directly from the server?
When this was tried the local project would not compile because it could not reach the server, but when the project is uploaded on the server it will need the correct file path
You could query against the json file directly on the server.
this.http.get('server_url/test/assets/json_file.json').subscribe(
data => {
//do what you want with data in here.
}
)
And to prevent your local server erroring, you could wrap the function call in environment.production == true or some such equivalent that prevent local.
In my development of Android and Java applications I have been using PHP scripts to interact with an online MySQL database, but now I want to migrate to Python.
How does one run Python scripts on a web server? In my experience with PHP, I have been saving my files under /var/www folder in a Linux environment. Then I just call the file later with a URL of the path. Where do I save my Python scripts?
You can use Flask to run webapps.
The simple Flask app below will help you get started.
from flask import Flask, jsonify
app = Flask(__name__)
#app.route('/sampleurl' methods = ['GET'])
def samplefunction():
#access your DB get your results here
data = {"data":"Processed Data"}
return jsonify(data)
if __name__ == '__main__':
port = 8000 #the custom port you want
app.run(host='0.0.0.0', port=port)
Now when you hit http://your.systems.ip:8000/sampleurl you will get a json response for you mobile app to use.
From within the function you can either do DB reads or file reads, etc.
You can also add parameters like this:
#app.route('/sampleurl' methods = ['GET'])
def samplefunction():
required_params = ['name', 'age']
missing_params = [key for key in required_params if key not in request.args.keys()]
if len(missing_params)==0:
data = {
"name": request.argv['name'],
"age": request.argv['age']
}
return jsonify(data)
else:
resp = {
"status":"failure",
"error" : "missing parameters",
"message" : "Provide %s in request" %(missing_params)
}
return jsonify(resp)
To run this save the flask app in a file e.g. myapp.py
Then from terminal run python myapp.py
It will start the server on port 8000 (or as specified by you.)
Flask's inbuilt server is not recommended for production level use. After you are happy with the app, you might want to look into Nginx + Gunicorn + Flask system.
For detailed instruction on flask you can look at this answer. It is about setting up a webserver on Raspberry pi, but it should work on any linux distro.
Hope that helps.
Use a web application framework like CherryPy, Django, Webapp2 or one of the many others. For a production setup, you will need to configure the web server to make them work.
Or write CGI programs with Python.
On Apache the simplest way would be to write the python as CGI here is an example:
First create an .htaccess for the web folder that is serving your python:
AddHandler cgi-script .py
Options +ExecCGI
Then write python that includes some some cgi libraries and outputs headers as well as the content:
#!/usr/bin/python
import cgi
import cgitb
cgitb.enable()
# HEADERS
print "Content-Type:text/html; charset=UTF-8"
print # blank line required at end of headers
# CONTENT
print "<html><body>"
print "Content"
print "</body></html>"
Make sure the file is owned by Apache chown apache. filename and has the execute bit set chmod +x filename.
There are many significant benefits to actually using a web framework (mentioned in other answers) over this method, but in a localhost web server environment set up for other purposes where you just want to run one or two python scripts, this works well.
Notice I didn't actually utilize the imported cgi library in this script, but hopefully that will direct you to the proper resources.
Most web development in python happens using a web framework. This is different than simply having scripts on the server, because the framework has a lot more functionality, such as handling URL routing, HTML templating, ORM, user session management, CSRF protection, and a lot of other features. This makes it easier to develop web sites, especially since it promotes component reuse, in a OOP fashion.
The most popular python web framework is Django. It's a fully-featured, tighly-coupled framework, with lots of documentation available. If you prefer something more modular and easier to customize, I personally recommend Flask. There's also lots of other choices available.
With that being said, if all you really want is to run a single, simple python script on a server, you can check this question for a simple example using apache+cgi.
Per suggestions on SO/SF and other sites, I am using CherryPy as the WSGI server to launch multiple instances of a Python web server I built with Flask. Each instance runs on its own port and sits behind Nginx. I should note that the below does work for me, but I'm troubled that I have gone about things the wrong way and it works "by accident".
Here is my current cherrypy.conf file:
[global]
server.socket_host = '0.0.0.0'
server.socket_port = 8891
request.dispatch: cherrypy.dispatch.MethodDispatcher()
tree.mount = {'/':my_flask_server.app}
Without diving too far into my Flask server, here's how it starts:
import flask
app = flask.Flask(__name__)
#app.route('/')
def hello_world():
return "hello"
And here is the command I issue on the command line to launch with Cherryd:
cherryd -c cherrypy.conf -i my_flask_server
Questions are:
Is wrapping Flask inside CherryPy still the preferred method of using Flask in production? https://stackoverflow.com/questions/4884541/cherrypy-vs-flask-werkzeug
Is this the proper way to use a .conf file to launch CherryPy and import the Flask app? I have scoured the CherryPy documentation, but I cannot find any use cases that match what I am trying to do here specifically.
Is the proper way to launch multiple CherryPy/Flask instances on a single machine to execute multiple cherryd commands (daemonizing with -d, etc) with unique .conf files for each port to be used (8891, 8892, etc)? Or is there a better "CherryPy" way to accomplish this?
Thanks for any help and insight.
I can't speak for Flask, but I can for CherryPy. That looks like the "proper way"...mostly. That line about a MethodDispatcher is a no-op since it only affects CherryPy Applications, and you don't appear to have mounted any (just a single Flask app instead).
Regarding point 3, you have it right. CherryPy allows you to run multiple Server objects in the same process in order to listen on multiple ports (or protocols), but it doesn't have any sugar for starting up multiple processes. As you say, multiple cherryd commands with varying config files is how to do it (unless you want to use a more integrated cluster/config management tool like eggmonster).
Terminology: Mounting vs Grafting
In principle this is a proper way to serve a flask app through cherrypy, just a quick note on your naming:
It is worth noting here that tree.mount is not a configuration key by itself - tree will lead to cherrypy._cpconfig._tree_config_handler(k, v) being called with the arguments 'mount', {'/': my_flask_server.app}.
The key parameter is not used at all by the _tree_config_handler so in your config "mount" is just an arbitrary label for that specific dict of path mappings. It also does not "mount" the application (it's not a CherryPy app after all). By that I mean, it does not cherrypy.tree.mount(…) it but rather cherrypy.tree.grafts an arbitrary WSGI handler onto your "script-name" (paths, but in CherryPy terminology) namespace.
Cherrypy's log message somewhat misleadingly says "Mounted <app as string> on /"]
This is a somewhat important point since with graft, unlike mount, you cannot specify further options such as static file service for your app or streaming responses on that path.
So I would recommend changing the tree.mount config key to something descriptive that does not invite reading too much semantics about what happens within CherryPy (since there is the cherrypy.tree.mount method) due to that config. E.g., tree.flask_app_name if you're just mapping that one app in that dict (there can be many tree directives, all of them just getting merged into the paths namespace) or tree.wsgi_delegates if you map many apps in that dict.
Using CherryPy to serve additional content without making an app of it
Another side note, if you want cherrypy to e.g. provide static file service for your app, you don't have to create a boilerplate cherrypy app to hold that configuration. You just have to mount None with the appropriate additional config. The following files would suffice to have CherryPy to serve static content from the subdirectory 'static' if they are put into the directory where you launch cherryd to serve static content (invoke cherryd as cherryd -c cherrypy.conf -i my_flask_server -i static:
static.py
import cherrypy
# next line could also have config as an inline dict, but
# file config is often easier to handle
cherrypy.tree.mount(None, '/static-path', 'static.conf')
static.conf
# static.conf
[/]
tools.staticdir.on = True
tools.staticdir.root = os.getcwd()
tools.staticdir.dir = 'static'
tools.staticdir.index = 'index.html'