Develop Reference

How delete a file in system32 folder with python?

for the thesis work on forensic analysis I should try to automatically delete a file created and placed in the "system32" folder.
This is the code I ran:
os.system("C://Windows//System32//update.exe")
os.chmod("C://Windows//System32//update.exe", stat.S_IRWXU)
os.remove("C://Windows//System32//update.exe")
The error is as follows:
Traceback (most recent call last): File "C:\Users\xxxx\PycharmProjects\Tesi\main.py", line 5, in <module> os.chmod("C://Windows//System32//update.exe", stat.S_IRWXU) PermissionError: [WinError 5] Accesso negato: 'C://Windows//System32//update.exe'
How can I run it with the right permissions?

Depends on how you are running the Python interpreter. If you are using Powershell, you can Right-click the Powershell icon and run it as Administrator. After this, any Python script you run from that shell will also run with admin rights.

Tensorflow load_weights() works in Google Colab but fails locally

I am working on a Tensorflow 2.0 project and am running into an interesting error. My project works with no errors when I run it in Google Colab. However, when I run the code locally, I get the following error:
ERROR:tensorflow:Couldn't match files for checkpoint followed by the file path to the appropriate checkpoint.
This error occurs when I call model.load_weights(tf.train.latest_checkpoint(CKPT_DIR)). Here is the traceback (on my Windows machine):
Traceback (most recent call last):
File ".\shakespeare_lstm.py", line 159, in <module>
run_model(SEED)
File ".\shakespeare_lstm.py", line 147, in run_model
model.load_weights(tf.train.latest_checkpoint(CKPT_DIR))
File "E:\Github Repos\ShakespeareLSTM\venv\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 181, in load_weights
return super(Model, self).load_weights(filepath, by_name)
File "E:\Github Repos\ShakespeareLSTM\venv\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 1139, in load_weights
if _is_hdf5_filepath(filepath):
File "E:\Github Repos\ShakespeareLSTM\venv\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 1449, in _is_hdf5_filepath
return (filepath.endswith('.h5') or filepath.endswith('.keras') or
AttributeError: 'NoneType' object has no attribute 'endswith'
This happens on my Windows machine that runs Python 3.6.8 and my MacOS machine that runs Python 3.7.2. The Google Colab runs Python 3.6.9 and throws no errors and executes as expected. All of them have Tensorflow 2.0.0.
My GitHub repo has the project code.
The only difference in the code that I have on Google Colab is the ROOT variable for the file paths (ROOT = os.path.join("/", "content", "drive", "My Drive", "Colab Notebooks", "Shakespeare LSTM")) and the following two additional lines (not directly after one another in the actual code):
from google.colab import drive
drive.mount('/content/drive')
Another difference is that the Google Colab notebook is set up to use a GPU, while my local machines are not. The model was trained on Google Colab, and the checkpoint files were downloaded.
Does anyone have any ideas as to why this code works on Google Colab and not locally?

osquery-python extension causing osqueryi errors

I have a newbie question about creating osquery extensions using osquery-python. I Created a small extension that gets some additional RPM info from my linux system. Following the instructions in the docs, I added the path to the extension in /etc/osquery/extensions.load to get it to autoload. I restarted osqueryd and I see the extension running using ps ax.
If I interactively run osqueryi, I can see the table and get data. It all works perfectly.
However, when I run an osqueryi command 'one-liner' such as :
osqueryi .tables
I get a bunch of the following errors with my output:
#INFO:thrift.transport.TSocket:Could not connect to /root/.osquery/shell.em
Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/thrift/transport/TSocket.py", line 104, in open
handle.connect(sockaddr)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory
ERROR:thrift.transport.TSocket:Could not connect to any of ['/root/.osquery/shell.em']
What have I done wrong?

Extensions are run in a separate process. You can see the socket errors, which indicate the extension process cannot communicate with osquery process. Make sure osqueryd or osqueryi is running. Link: osquery doc page for extensions.

"No space left on device" when installing Numpy on an Azure Web Role

I want to use Numpy and Pandas in a Python Web Role on Microsoft Azure.
I tried adding numpy and pandas to the requirements.txt, but this didn't work (using pip to install numpy often leads to problems so this was expected).
I followed this advice and downloaded numpy as a wheel from http://www.lfd.uci.edu/~gohlke/pythonlibs/ and put the file in the root of the web role. Right now the requirements.txt looks like this:
azure>=0.8.0
azure-storage-logging
requests_futures
numpy-1.9.3+mkl-cp34-none-win32.whl
pandas
I remoted into the virtual machine and found some logs in C:\Resources\Directory\7044b9f2b424470aa191d9c178d06399.WorkerRole1.DiagnosticStore\LogFiles\ConfigureCloudService:
Storing debug log for failure in D:\Windows\system32\config\systemprofile\pip\pip.log
pip 1.5.6 from D:\Python34\lib\site-packages (python 3.4)
Unpacking e:\approot\numpy-1.9.3+mkl-cp34-none-win32.whl
Cleaning up...
Exception:
Traceback (most recent call last):
File "D:\Python34\lib\site-packages\pip\basecommand.py", line 122, in main
status = self.run(options, args)
File "D:\Python34\lib\site-packages\pip\commands\install.py", line 278, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "D:\Python34\lib\site-packages\pip\req.py", line 1197, in prepare_files
do_download,
File "D:\Python34\lib\site-packages\pip\req.py", line 1364, in unpack_url
unpack_file_url(link, location, download_dir)
File "D:\Python34\lib\site-packages\pip\download.py", line 640, in unpack_file_url
unpack_file(from_path, location, content_type, link)
File "D:\Python34\lib\site-packages\pip\util.py", line 621, in unpack_file
unzip_file(filename, location, flatten=not filename.endswith(('.pybundle', '.whl')))
File "D:\Python34\lib\site-packages\pip\util.py", line 510, in unzip_file
fp.write(data)
OSError: [Errno 28] No space left on device
Storing debug log for failure in D:\Windows\system32\config\systemprofile\pip\pip.log
How can I get Numpy working?

According your description, it looks like you host your python application on Azure Could Service. If you use Visual Studio as your IDE, I suggest that you can follow this solution:
You can try to set python virtual environment as python runtime in Web Role package, and leverage Visual Studio we can configure and deploy cloud service in python with a ease.
1, In your Azure Cloud Service project solution, right click Python Environment under your web role package, click add virtual environment.
2, If you named the virtual env as env, VS will create a VE folder named env in your web role root directory. Copy the wheel file numpy-1.9.3+mkl-cp34-none-win32.whl into env folder.
3, configure requirements.txt, to completely configure the dependencies of pandas and numpy, here is my requirements.txt looks like beside azure package:
pandas
numpy-1.9.3+mkl-cp34-none-win32.whl
six
pytz
4, right click the virtual env in the solution tree, click Install from requirements.txt to install all the packages.
5, then deploy it to Azure
It works fine via this method on my side. Any concern, please feel free let me know.

In my case, it was caused by Drive C: is too full. All the temporary file for all of the process in your PC is stored in Drive C:, so you can't install big package when it's full.

I've been working through this for about 3 days and after debugging the pip code and other bits and pieces my fix is:
iex "$env:PYPATH\Scripts\pip.exe install -b e:\approot\stuff -r requirements.txt"
notice the -b.
for some reason Windows/Python thought it had run out of room in the standard temp directory: c:\resources\app_id..... (when there was > 200GB full)
by using the -b it forces pip to use your own Build directory which doesn't fill up.
sohappy

I had the same problem with installing Numpy.
The base template for my Azure Web role I took from this article.
As an idea to fix this error I took from Gordon's answer.
I added tmp Local Storage to my RoleProperties:
After that made a change to the Startup task in the ServiceDefinition.csdef. Added a variable for the "tempdir":
<Task executionContext="elevated" taskType="simple" commandLine="bin\ps.cmd
PipInstaller.ps1">
<Environment>
<Variable name="EMULATED">
<RoleInstanceValue xpath="/RoleEnvironment/Deployment/#emulated" />
</Variable>
<Variable name="TEMPDIR">
<RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/LocalResources/LocalResource[#name='tempdir']/#path" />
</Variable>
<Variable name="PYTHON2" value="off" />
</Environment>
</Task>
The final step. Changed "PipInstaller.ps1" to force pip to use "tempdir" as a Build directory:
$tmp_dir = $env:TEMPDIR
py -m pip install -b $tmp_dir -r ..\requirements.txt

PermissionError: [WinError 5] Access is denied python using moviepy to write gif

I'm using windows 8.1 64 bit
my code
import pdb
from moviepy.editor import *
clip = VideoFileClip(".\\a.mp4")
clip.write_gif('.\\aasda.gif')
the exception is at write_gif method
Traceback (most recent call last):
File "C:\abi\youtubetogif_project\test.py", line 5, in <module>
clip.write_gif('G:\\abi\\aasda.gif')
File "<string>", line 2, in write_gif
File "C:\Python34\lib\site-packages\moviepy-0.2.1.8.12-py3.4.egg\moviepy\decorators.py", line 49, in requires_duration
return f(clip, *a, **k)
File "C:\Python34\lib\site-packages\moviepy-0.2.1.8.12-py3.4.egg\moviepy\video\VideoClip.py", line 435, in write_gif
dispose= dispose, colors=colors)
File "<string>", line 2, in write_gif
File "C:\Python34\lib\site-packages\moviepy-0.2.1.8.12-py3.4.egg\moviepy\decorators.py", line 49, in requires_duration
return f(clip, *a, **k)
File "C:\Python34\lib\site-packages\moviepy-0.2.1.8.12-py3.4.egg\moviepy\video\io\gif_writers.py", line 186, in write_gif
stdout=sp.PIPE)
File "C:\Python34\lib\subprocess.py", line 848, in __init__
restore_signals, start_new_session)
File "C:\Python34\lib\subprocess.py", line 1104, in _execute_child
startupinfo)
PermissionError: [WinError 5] Access is denied
I moved the script to another folder and partition, running moviepy dependancies and python as admin, turning off UAC still gives me error

I've run into this as well, solution is usually to be sure to run the program as an administrator (right click, run as administrator.)

Sometimes it occurs when some installations are not completed correctly, the process is stuck, or a file is still opened. So, when you try to run the installation again and the installation requires deleting, you can see the aforementioned error. In my case, shutting down the python processes and command prompt utilization helped.

this resolved my problem
Click on the search button in the taskbar and type “cmd”. Right-click on the Command Prompt and select Run as Administrator
pip install pydirectory

Solution on windows : restarted docker
On windows I used --use-container option during sam build
So, in order to fix stuck process, I've restarted docker

I got the same error when an imported library was trying to create a directory at path "./logs/".
It turns out that the library was trying to create it at the wrong location, i.e. inside the folder of my python interpreter instead of the base project directory. I solved the issue by setting the "Working directory" path to my project folder inside the "Run Configurations" menu of PyCharm. If instead you're using the terminal to run your code, maybe you just need to move inside the project folder before running it.

If you're encountering this in Jupyter/Jupyerlab while trying to pip install foo, you can sometimes work around it by using !python -m pip install foo instead.

On Windows, for me, it looked like at some point I'd set the folder to read-only.
Not really sure when, possibly during some mount failure on my Linux boot, but recursively clearing that flag helped.

I know it is pretty old and a couple of fellows has given the abstract answer to it.
But this is how I solved this problem on my machine. (Thanks #DevLoverUmar and #Vladyslav Didenko)
pip install gym --user

This might happen when the working directory path is different from the where the file is present . For example while running files and importing them in Spyder3 I have to check the working directory .

Maybe you wrongly set permission on python3. For instance if for the file permission is set like
`os.chmod('spam.txt', 0777)` --> This will lead to SyntaxError
This syntax was used in Python2. Now if you change like:
os.chmod('spam.txt', 777) --> This is still worst!! Your permission will be set wrongly since are not on "octal" but on decimal.
Afterwards you will get permission Error if you try for instance to remove the file:
PermissionError: [WinError 5] Access is denied:
Solution for python3 is quite easy:
os.chmod('spam.txt', 0o777) --> The syntax is now ZERO and o "0o"