Integrating Docker with PyCharm : file not found? - python

I am currently trying to set up a workflow to build Pytorch + Docker container such that I can train models elsewhere. I got the Pytorch image from the Nvidia registry and am trying to integrate docker into PyCharm, where my code currently is. Now I'm a bit confused. It is apparently possible to connect with docker and build containers which can be used as python interpreter (https://www.jetbrains.com/help/pycharm/using-docker-as-a-remote-interpreter.html). My idea is to get everything running and then export it again as image. I got the Pytorch container running as described but receive following error when I try to run code:
File "/opt/.pycharm_helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/opt/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 11, in execfile
stream = tokenize.open(file) # #UndefinedVariable
File "/opt/conda/lib/python3.8/tokenize.py", line 392, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: /home/...
I tried setting paths in the interpreter settings but that did not help. What am I doing wrong? I want to test code in the container and in the end get an image with everything. Is there a better way to do that?

Related

Issues when launching a dash application packaged with Pyinstaller

Situation:
I have created a simple dash Cytoscape application, compile it into an executable file using Pyinstaller and tried running it on different computers in the same network all without issues. However, when I tried running it on a different network, I got back an error msg as follows:
File "dash\dash.py", in run_server
self.server.run(host=host, port=port, debug=debug, **flask_run_options)
File "flask\app.py, in run
File "werkzeug\serving.py, in run_simple
File "werkzeug\serving.py, in inner
File "werkzeug\serving.py, in make_server
File "werkzeug\serving.py, in __init__
File "socketserver.py", in __init__
File "socket.py", in __init__
OSError: [WinError 10022] An invalid argument was supplied
Failed to execute script 'Application' due to unhandled exception!
Dash Cytoscape Application script that returns an error when running the executable file:
def open_browser():
webbrowser.open_new('http://127.0.0.1:5000/')
if __name__=='__main__'
Timer(2,open_browser).start()
app.run_server(port=5000, debug=False)
Just wondering why the other computer on the same network is able to launch the executable file but not those on another network? To add on the port 5000 is not in use. Hope to get some clarity and direction on StackOverflow. Appreciate all help.

access mounted data in jupyter notebook on amazon ec2

I am trying to run a jupyter notebook using data and notebooks that are mounted on an EBS volume on my ec2 instance. My ec2 instance uses ubuntu. My directory structure looks like the following:
/
---|mountedData
---|localData
I used the instructions provided here here to set up the notebook. When I invoke the jupyter notebook command from / or from /localData it is successful. However, I can't navigate to the /mountedData directory (it doesn't even show up on the browser's file navigation screen). If I launch the jupyter notebook from within /mountedData I get an error in the browser
Server error: Traceback (most recent call last): File "/snap/jupyter/6/lib/python3.7/site-packages/tornado/web.py", line 1699, in _execute result = await result File "/snap/jupyter/6/lib/python3.7/site-packages/tornado/gen.py", line 209, in wrapper yielded = next(result) File "/snap/jupyter/6/lib/python3.7/site-packages/notebook/services/contents/handlers.py", line 112, in get path=path, type=type, format=format, content=content, File "/snap/jupyter/6/lib/python3.7/site-packages/notebook/services/contents/filemanager.py", line 431, in get model = self._dir_model(path, content=content) File "/snap/jupyter/6/lib/python3.7/site-packages/notebook/services/contents/filemanager.py", line 313, in _dir_model for name in os.listdir(os_dir): PermissionError: [Errno 13] Permission denied: '/var/lib/snapd/void'`
All directories are owned by root and root is the usergroup. I even tried chmod 777'ing the /mountedData but that didn't help. I also tried symlinking the mounted data within /localData as I saw suggested online but that produces a 404 not found error when I try to click on the symlink. Unfortunately ditching the mounted data is not an option as I am working with TBs of data that I need to mount and attach to ec2 instances. Thanks for the help!
I managed to solve this problem by uninstalling the version of jupyter I had installed via snap, then reinstalling jupyter via anaconda. So the moral of the story is to use anaconda to install python packages!

OSError: [WinError 193] %1 is not a valid Win32 application error using GeckoDriver and Firefox through Selenium and Python on Windows

I created script with selenium in Ubuntu and works just fine there, but when I moved it to windows10, I get lots of error and I tried to fix it one by one until I see this error. I've been looking for the solution to this problem but I am unable to resolve this error.
Traceback (most recent call last):
File "D:/Users/b/Documents/Python/Bolt/GUI.py", line 180, in start
driver = l.start_chime() # start chime
File "D:\Users\b\Documents\Python\Bolt\Login.py", line 87, in start_chime
self.chime_driver = webdriver.Firefox(executable_path=self.PATH)
File "D:\Users\b\Documents\Python\Python3.8\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 164, in __init__
self.service.start()
File "D:\Users\b\Documents\Python\Python3.8\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "D:\Users\b\Documents\Python\Python3.8\lib\subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "D:\Users\b\Documents\Python\Python3.8\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
File "C:\Program Files\JetBrains\PyCharm 2020.1.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_monkey.py", line 551, in new_CreateProcess
return getattr(_subprocess, original_name)(app_name, patch_arg_str_win(cmd_line), *args)
OSError: [WinError 193] %1 is not a valid Win32 application
This is happen when I tried to open webdriver using selenium.
self.myday_driver = webdriver.Firefox(executable_path=self.PATH)
and is there any method to move script from Ubunto to Windows without getting error?
I will try to help you answer your last question:
and is there any method to move script from Ubunto to Windows without
getting error?
Yes, have you heard about docker? https://www.docker.com/
Essentially, docker will create isolated environments that will run in every machine that has docker installed. These environments are configurable inside a dockerfile, basically, you need to follow these steps:
Install docker on both machines. I have used on Windows and RH to automate all this process and minimize erros.
Create a docker file, the structure will be something like:
FROM ubuntu:18.04
COPY . /app
RUN make /app
CMD python /app/app.py
So it will create an environment based on a ubuntu image
copy all files in your current directory (.) to /app (remember this will be a ubuntu image, so you have a standard folder structure with /etc /home, etc.)
run command make (in your case could be install some dependency using pip)
run python command.
You also can find python images ready for usage, so instead of ubuntu:latest you could get a image linux with python installed and then you just install your dependencies.
This is a great tool for a developer, I recommend looking into it, read documentation to understand concepts and it will ease your life.
Hope it helps.
This error message...
OSError: [WinError 193] %1 is not a valid Win32 application
...implies that the underlying OS doesn't recognizes %1 i.e. the system variable PATH as a valid Win32 application i.e. a executable binary.
To initiate a Selenium driven GeckoDriver controlled Firefox session you need to:
Download the latest version of GeckoDriver binary version, place it in your system.
Next in your code block you need to mention the absolute path of the binary through the Key executable_path as follows:
from selenium import webdriver
self.myday_driver = webdriver.Firefox(executable_path=r'C:\path\to\geckodriver.exe')

How to run batch file through Windows Scheduler running python file outputting plotly plot

I have a batch file which runs a python file outputting an interactive plot via Plotly in an html file. Running the file via double-clicking runs just fine, but when running through Windows scheduler, it doesn't work.
Python code producing plotly html:
plot(fig,filename='output.html')
Batch file running python file:
python file.py
Double-clicking on batch file works.
Running batch file via Window's Scheduler doesn't work and outputs the following error in the command prompt:
Traceback (most recent call last):
File "C:\~\file.py", line 224, in <module> plot(fig,filename='output.html')
File "C:\ProgramData\Anaconda3\lib\site-packages\plotly\offline\offline.py", line 721, in plot with open(filename, 'w') as f:
PermissionError: [Errno 13] Permission denied: 'output.html'
Any idea why this won't work via Windows Scheduler?
Figured it out.
I hadn't set the task in the Task Scheduler to "Run with highest privileges." Checking that box fixed the issue.

Is it possible to use Ropemacs with TRAMP in Emacs?

I recently installed the hg tip version of Ropemacs and I'd like to use it when editing remote files using TRAMP. Has anyone done this? When I try to use M-/ to complete a variable name, I am asked to enter the Rope project root folder and I enter: /ssh:myhost:/path/to/myproject/ and it gives me the following error:
Opening [/ssh:myhost:/path/to/myproject/] project ...
pymacs-report-error: Python: Traceback (most recent call last):
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/Pymacs/pymacs.py", line 147, in loop
value = eval(text)
File "<string>", line 1, in <module>
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/decorators.py", line 53, in newfunc
return func(*args, **kwds)
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/interface.py", line 254, in code_assist
_CodeAssist(self, self.env).code_assist(prefix)
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/interface.py", line 538, in code_assist
proposals = self._calculate_proposals()
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/interface.py", line 597, in _calculate_proposals
self.interface._check_project()
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/interface.py", line 448, in _check_project
self.open_project()
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/decorators.py", line 53, in newfunc
return func(*args, **kwds)
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/ropemode/interface.py", line 88, in open_project
self.project = rope.base.project.Project(root)
File "/home/saltycrane/lib/python-environments/default/lib/python2.6/site-packages/rope/base/project.py", line 134, in __init__
os.mkdir(self._address)
OSError: [Errno 2] No such file or directory: '/ssh:myhost:/path/to/myproject'
I imagine ropemacs doesn't support remote files with TRAMP. Just wondering if anyone has found a workaround. Maybe some custom elisp? Or maybe I should use sshfs and mount the remote filesystem locally...
I installed ropemacs in a virtualenv located at /home/saltycrane/lib/python-environments/default. I am using GNU Emacs 23.1.50.1 on Ubuntu Lucid.
I should mention that completion with M-/ works fine when I'm not using TRAMP.
No, this is not possible using TRAMP, because the Python rope library does not make calls back into Emacs when it wants to scan the filesystem for Python files to find their classes and functions — instead, it uses calls like listdir() and open() on the filesystem directly, without giving Emacs a chance to get in the way and intercept the special TRAMP-prefixed paths that you are editing.
Using sshfs might be possible, as the comments have mentioned, but for rope I imagine that it would be a very slow solution, since rope has to open so many files to find out the state of your project.
I generally leave rope off, or else figure out how to export an Emacs session from the remote system, when editing across a remote-filesystem link. I wonder if you could use something like rsync or dropbox or even a DVCS so that the files would be present on your local hard drive as immediately-accessible copies, but so that changes you save would get copied across?

Categories

Resources