Develop Reference

Does anyone know why I get a file not found error in Jenkins when opening a JSON file with Python (Locust)?

I'm using Locust for load testing - creating a lot of post requests to a server.
Because I need to generate different fields for every request, The best way to do it in my opinion is to read the body from a file, change the relevant fields and send the request.
The problem occurs when I open the file
I see in the Jenkins log that there is a FileNotFound exception - even though I see the file in the git repo from where the Jenkins runs the code.
I tried putting the full path in the with statement but still got the same exception.
...
with open('postRequest.json', 'r') as jsonFile:
data = json.load(jsonFile)
data["a"] = b
data["x"] = y
data[["something"] = something_else
return json.dumps(data)
Jenkins fails opening the file.
Note : The code works when I don't read the file, but just create a very long JSON string.
Thanks all !! ;)

The issue was resolved, in the Jenkins the full path is different than I thought it was.
Anyway, ran pwd and saw where I was - added the path where the file was and worked.
Thanks friends !

Execute python script in Qlik Sense load script

I am trying to run python script inside my load script in Qlik Sense app.
I know that I need to put OverrideScriptSecurity=1 in Settings.ini
I put
Execute py lib://python/getSolution.py 100 'bla'; // 100 and 'bla' are parameters
and I get no error in qlik sense, but script is not executed (I think) because inside the script I have
f = open("file.xml", "wb")
f.write(xml)
f.close
and file is not saved.
If I run script from terminal, then script is properly executed.
What could go wrong?
By the way, my full path to python interpreter is
C:\Users\Marko Z\AppData\Local\Programs\Python\Python37-32\python.exe
EDIT :
Even if I add this
Set vPythonPath = "C:\Users\Marko Z\AppData\Local\Programs\Python\Python37-32\python.exe";
Set vPythonFile = "C:\Users\Marko Z\Documents\Qlik\Sense\....\getSolution.py";
Execute $(vPythonPath) $(vPythonFile);
I get the same behaviour. No error, but not working,...
I even see that if I change path (incorrect path) it give me an error, but incorrect file it doesn't give me an error.... (but I am sure it is the right file path...)
My python code is
xml = "Marko"
xml = xml.encode('utf-8')
f = open("C:\\Users\\Marko Z\\Test.xml", "wb")
f.write(xml)
f.close

I figure out what was wrong.
For all others that would have similar problems:
Problem is in space in path.
If I move my script in c:\Windows\getSolution.py it work. I also need to change the python path to c:\Windows\py.exe
so end script looks like:
Execute c:\Windows\py.exe c:\Windows\getSolution.py 100 'bla';
But I still need to figure how to work with space in path...

Strange. With exactly the same python file and QS script the result file is generated correctly.
The content of my settings.ini.
[Settings 7]
StandardReload=0
OverrideScriptSecurity=1
According to Qlik's documentation there should be an empty line at the end (point 4 from the lists)

I was able to get the below to work in qlik sense:
set vPyExe = C:\Program Files\Python37\python.exe;
set vPyScript = D:\...\PythonScript.py;
Execute
"$(vPyExe)" "$(vSource)"
;

Python script not writing to txt file

I have a Python script that runs properly on my laptop, but when running on my raspberry pi, the following code does not seem to be working properly. Specifically, "TextFile.txt" is not being updated and/or saved.
openfile = open('/PATH/TextFile.txt','w')
for line in lines:
if line.startswith(start):
openfile.write(keep+'\n')
print ("test 1")
else:
openfile.write(line)
print ("test 2")
openfile.close()
I am seeing "test 1" and "test 2" in my output, so I know that the code is being reached, paths are correct, etc
It may be due to a permissions problem. I am running the script from the terminal by using:
usr/bin/python PATH/script.py
Python is owned by "root" and script.py is owned by "Michael".

My first guess:
Does the file exist? If it does not exist then you cannot write to it. Try this to create the file if it does not exist: file = open('myfile.dat', 'w+')
Additionally manually opening and closing file handles is bad practice in python. The with statement handles the opening and closing of the resource automatically for you:
with open("myfile.dat", "w+") as f:
#doyourcalculations with the file object here
for line in f:
print line

All, thank you for your input. I was able to figure out that it was writing to the new file, but it was overwriting with the same text. The reason was because ".startswith" was returning false when I expected true. The misconception was due to the difference between how Windows and Unix treat new line characters (/n /r).

Since your code is running, there should be a file somewhere.
You call "PATH/script.py", but there is "/PATH/TextFile.txt" in your program. Is the slash before PATH a mistake? Have you checked the path in your program is really where you are looking for the output file?

python astonishing IOError on windows creating files - Errno 13 Permission

I have to run my python script on windows too, and then it began the problems.
Here I'm scraping html locally saved files, and then saving their .csv versions with the data I want. I ran it on my ubuntu and goes for +100k files with no problems. But when I go on windows, it says:
IOError: [Errno 13] Permission denied
It is not a permissions problems, I've rechecked it, and run it under 'Administration' powers, and it makes no difference.
It breaks exactly on the line where I open the file:
with open(of, 'w') as output:
...
I've tried to create same first file of the 100k from the python console and from a new blank stupid script from same directory as my code, and it works...
So, it seems is doable.
Then I've tried with output = open(of, 'w') instead of above code but nothing.
The weird thing is that it creates a directory with same name as the file, and then breaks with the IOError.
I've started thinking that it could be a csv thing..., naaaeehh, apart from other tries that didn't helped me, the most interesting stuff is that with the following code:
with open(of+.txt, 'w') as output:
...
it happens the astonishing thing that it creates a directory ending on .csv AND a file ending in .csv.txt with the right data!
Aargh!
Changing the open mode file to 'w+', 'wb', it didn't make a difference either.
Any ideas?

You can get permission denied if the file is opened up in another application.
Follow this link to see if any other process is using it: http://www.techsupportalert.com/content/how-find-out-which-windows-process-using-file.htm
Otherwise, I would say to try to open the file for read instead of write to see if it allows you to access it at all.
-Brian

Damn it, it's already working!, it has been like saying i cannot find my glasses and to have them on.
THanks Brian, it wasn't that the error. The problem was that in my code i was dealing with ubuntu separator besides the full path to the csv output file was completely correct. But I replaced it with os.sep , and started working like a charm :)
Thanks again!

How do I mark a .txt file as "not executable"?

I am hosting a text file on OpenShift inside the cron/minutely directory. The server is trying to run the text file every minute, only I don't want it to: all I want the text file to do is stay on the server so I can save useful information inside it.
How do I tell the server that I want it to ignore the text inside the file instead of running it as code? Is there a way to do it by adding a specific "shebang" line at the beginning? Is it even possible to keep the text file in the cron/minutely directory without it being executed?
Thank you.

You can add this as the first line of the text file:
#!/usr/bin/true
That's a sort of fake shebang line. It means that if the file is executed, the "interpreter" to "run" the file is the true program, which is a trivial program which does nothing but return the status code for success (0).
Therefore, a true shebang will make any file "successfully" execute, but do nothing.
Please don't take this to mean that leaving data files in a directory meant for executables is a good idea; it's not. This is just a hack of last resort.