I have a Django app deployed on a Azure Web App, and I want to dynamically create webjobs. More precisely, when I save a Django model called Importer, I want to be able to create a new web job, made of 3 files:
run.py : the Python webjob
settings.job : the cron schedule
config.ini : a config file for the webjob
The content of the "settings.job" & "config.ini" comes from an admin form and is specific to the webjob.
When I save my model, the code creates a new directory in
App_Data\jobs\triggered{my job name}
, and copies there the "run.py" from my app directory.
This works. However, when I try to create a new text file called "settings.job" in the job directory and write the cron schedule in it, I got a server error.
I tried many things, but the following basic test causes a failure:
file = open('test.txt','w')
file.write('this is a test')
file.close()
It seems that I don't have the right to write a file to the disk. How can that be solved?
Also, I want to be able to modify the content of the config and settings.job files when I update the corresponding Django model.
In addition, I tried to copy another file called "run2.py" to the webjob directory, and that fails too ! I cannot copy another file that run.py in that directory
According to your description, per my experience, I think the issue was caused by using the relative path in your code.
On Azure WebApps, we have the permission of doing any operations under the path D:/home.
My suggestion is using the absolute path instead of the relative path, such as D:/home/site/wwwroot/App_Data/jobs/triggered/<triggered-job-name>/test.txt or /home/site/wwwroot/App_Data/jobs/triggered/<triggered-job-name>/test.txt instead of test.txt. And please make sure the directory & parent directories had been made via use os.path.exists(path) to check the path exists and use os.makedirs to make them before writing a file.
Meanwhile, you also can try to use the WebJobs API to do some operations, such as creating a webjob via uploading a zip file or updating settings.job. For using these WebJobs APIs, be sure you had configured the Deployment credentials on Azure portal as the figure below, and add the basic auth Authorization: Basic <BASE64-Encode("deployment-username":"password")>in the request header.
Here is my analysis:
there is no problem copying other files than "run.py" to the webjob directory
the code crashes after (successfully) copying the file "run.py" from my Django app directory to the webjob directory. It does not matter if I use shutil.copy/shutil.copyfile or simply open("{path}/run.py","w"), the problem occurs when I try to write a file called "run.py" to the disk in the webjob directory.
I think that when we create a new webjob directory, if the system detects a file called "run.py" it tries to carry out some operations. Then there is a conflict with two processes trying to access the same file at the same time.
My solution: I copy the python script to the webjob directory with the name "myrun.txt", and then I rename it to run.py using os.rename
It works !
Related
We have a python application on a window azure VM that reads data from an API and loads the data into an onprem DB. The application works just fine and the code is source controlled in an Azure devops repo. The current deployment process is for someone to pull the main branch and copy the application from their local machine to c:\someapplication\ on the STAGE/PROD server. We would like to automate this process. There are a bunch of tutorials on how to do a build for API and Web applications which require your azure subscription and app name (which we dont have). My two questions are:
is there a way to do a simple copy to the c:\someapplication folder from azure devops for this application
if there is, would the copy solution be the best solution for this or should I consider something else?
Should i simply clone the main repo to each folder location above and then automate the git pull via the azure pipeline? Any advice or links would be greatly appreciated.
According to your description, you could try to use the CopyFiles#2 task and set the local folder as shared folder, so that use it as TargetFolder. The target folder or UNC path that will contain the copied files.
YAML like:
- task: CopyFiles#2
inputs:
SourceFolder:$(Build.SourcesDirectory) # string. Source Folder.
Contents: '**' # string. Required. Contents. Default: **.
TargetFolder: 'c:\\someapplication' # string. Required. Target Folder.
Please check if it meets your requirements.
I have been experimenting with the Python Cloud Functions. One of my cloud functions utlizies a large text file, that I would love to bundle with my .py file when I deploy it. THe docs are kinda limited on stuff like this..
https://cloud.google.com/functions/docs/quickstart
I was looking, would I just include that file, and a requirements file in my same directory as my function to deploy it. Or do I have to some how require it in my code?
Also, is there any information on how to use a database trigger instead of a http trigger? I was trying to think if the reason my file didn't seem to get included was because I had the wrong way of defining the trigger. How would you create an OnCreate.. or something like that
gcloud beta functions deploy hello_get --runtime python37 --trigger-http
When you deploy your function using gcloud, your project directory is zipped and uploaded. Any files contained in the current directory (and child directories) will be uploaded as well, which includes static assets like your large text file.
Just make sure your text file is in your project directory (or one of the child directories), then use a relative reference to your file in your Python code. It should just work.
I'm looking for a functional build definition for a Django 2.0.5 app in Visual Studio Team Services.
What I'm trying to do is deploying my Django application to Azure App Services trough CI/CD, and the process is running well (I mean, the files is being traspased to production), but, for some reason, my app doesn't starts and I'm getting a lot of errors. I guess I'm traspassing the wrong files, but I really don't know it.
Can someone please tell me what did you do to build a Django app with CI/CD in VSTS?
EDIT:
What I'm doing is use a default build definition for Python called "Python package", but customized. In one of the build steps, it calls to "setup.py" file. I created the file in the root folder of my repository, and I wrote the next code:
import os
import shutil
from distutils.dir_util import copy_tree
if not (os.path.exists('./dist')):
os.makedirs('dist')
else:
shutil.rmtree('./dist')
copy_tree(".", "./dist")
Ok, I know the code is ugly, and I need to write it in a better way, but is functional and it copy the entire project structure into the "./dist" directory. Later, the release proccess takes all these files and copy it to production environment. All this works Ok, but my app never starts. When I try to navigate my url, I receive a 404 error, and can't use any of the project routes.
Hope it helps, if not, I will put all the necessary details, thank's again.
Finally I found the problem. I didn't traspased the environment folder, and I has missed web.config file too.
Uploading the env folder and the web.config file fix the application.
ref: https://github.com/azureappserviceoss/djangoapp
I am working on a Django based application whose location on my disk is home/user/Documents/project/application. Now this application takes in some values from the user and writes them into a file located in a folder which is under the project directory i.e home/user/Documents/project/folder/file. While running the development server using the command python manage.py runserver everything worked fine, however after deployment the application/views.py which accesses the file via open('folder/path','w') is not able to access it anymore, because by default it looks in var/www folder when deployed via apache2 server using mod_wsgi.
Now, I am not putting the folder into /var/www because it is not a good practise to put any python code there as it might become readable clients which is a major security threat. Please let me know, how can I point the deployed application to read and write to correct file.
The real solution is to install your data files in /srv/data/myapp or some such so that you can give the webserver user correct permissions to only those directories. Whether you choose to put your code in /var/www or not, is a separate question, but I would suggest putting at least your wsgi file there (and, of course, specifying your <DocumentRoot..> correctly.
Is there an easy way to edit our python files in the Jenkins workspace UI?
It would be super nice if we could get code highlighting too!
There is a jenkins plugin that allows you to edit files: Config File Provider
It cant edit random file but you can use it to achieve what you want.
The storage of the plugin is in the form of xml files in jenkins folder. This means that you could create script that recreates those files wherever you need them by parsing those xml files (plugin does this for the workspace although it requires build setp). For instance, i could add new custom config file like this:
Name: script.sh
Comment: /var/log
Content: ....
This will be available then in xml file which you could parse within cron job to create actual files where you need them
The closest I can think of that Jenkins offers is a file upload. You can upload file with local changes and then trigger a build. This file will be replaced at already specified location. This feature can be used by making your build parameterized and adding File Parameter option. Below is what Jenkins says about the description of this feature.
Accepts a file submission from a browser as a build parameter. The uploaded file will be placed at the specified location in the workspace, which your build can then access and use.
This is useful for many situations, such as:
Letting people run tests on the artifacts they built.
Automating the upload/release/deployment process by allowing the user to place the file.
Perform data processing by uploading a dataset.
It is possible to not submit any file. If it's case and if no file is already present at the specified location in the workspace, then nothing happens. If there's already a file present in the workspace, then this file will be kept as-is.