I'll preface this with that until last week, my entire experience with linux can be summarized by some long-forgotten class in school 10 years ago, plus my recent use of Git Bash on Windows. Bear with me!
I've recently started working with GCP to do data-sciency stuff. I created a VM instance to load and process a bunch of data using a Python script I wrote.
However, when I start the instance, I see that it's using Python 2.7, so I naturally tried to figure out how to install or update Python. www.python.org said that I'd likely have to build the library from source code, so that's what I learned how to do (thanks SO).
Now, as I'm trying to run my script, I keep getting errors due to missing modules. A little research told me that these are core library modules that are missing... As in they should be part of my Python environment already. So I keep pip installing these missing modules, but each and every time, I have to re-build Python from scratch in order for the newly installed modules to be recognized.
Let's just say, that's not fun for a Windows los... user.
So, how do I get a Python distribution that has all the regular libraries already installed, like I would on Windows?
Related
I'm looking for a way to call my python project and display the console on my c# app.
My python project is a bit particular, i use some specific librairy, i even have to install a package whl not avalible with pip, avalible only with python3.7(not with 3.8) etc. So to run the project, i need python 3.7 exactly.
The problem is also that i want to deploy my c# app in ClickOnce to be use by clients, without having them to install a local python version.
I've seen on the net two ways to do work with python and c#, that both doesn't seem working for my need.
using python in c# app
1.Call python in a shell
I've imported the python project in my c# app, i've called the python.exe avalible in the venv and i've deployed the app. Everithing seems to work, but i discover that python executable in the venv refer to the local python installation and doesn't seem autonomus. So it was only working for me and not the clients.
Use IronPython in c#
Really far from working, starting by the encoding blowing from everywhere, even in imported librairy like numpy. I've referenced my venv librairy in SetSearchPaths(), and even with that, it's doesn't seem to work.
Any suggestions? The best way i think, would be to have a python.exe independant following the project that can load my venv.
Python.Included and the Numpy.NET package made with it may work for this:
https://github.com/henon/Python.Included
https://github.com/SciSharp/Numpy.NET
They are introduced in this post from 2019 and seem to be in active development today, https://medium.com/scisharp/using-python-libraries-in-net-without-a-python-installation-11124d6190cf
That solution does not use IronPython, but Python.NET: https://github.com/pythonnet/pythonnet
IronPython is an implementation of Python in C#, championed by others and then Microsoft itself back in 2010 era. Later Microsoft dropped it and largely the whole notion of supporting dynamic languages on .NET (they made a DLR system for it back then). It is very cool and works for pure Python code.
But NumPy and many useful and popular Python modules are written in C, using the C API of Python.org 's default C written Python implementation, aka. CPython. This is what Microsoft decided to back too instead, because C written Python modules don't work (easily and well) with IronPython. Also IronPython remains at Python 2.7.
Python.NET just bridges the .NET land with the normal CPython interpreter, so that you can call code cross the language boundary. So Numpy and everything works the same as with Python usually. Which is what you want.
Python.Included is one way to deploy that in C# projects - which may or may not work for you, but at least provides a starting point:
Python.Included is an automatic deployment mechanism for .NET packages
which depend on the embedded Python distribution. This allows
libraries depending on Python and/or Python packages to be deployed
via Nuget without having to worry about any local Python
installations.
It packages embedded Python (python-3.7.3-embed-amd64.zip) in its .NET
assembly and automatically deploys it in the user's home directory
upon first execution. On subsequent runs, it will find Python already
deployed and therefor doesn't install it again.
If you don't want to use that install mechanism, I think you can just bundle the CPython interpreter with your C# application and use the Python.NET mechanism to call that from your app's directory instead.
I am trying to debug an application that is written in Python 2. Problem is, before application can be run, it must be installed (according to the documentation) like this:
python setup.py develop
Application (Cuckoo Sandbox) then can be run like this:
cuckoo -d
With the new Python package developing and testing code now works slightly different than it used to be. As one
will first have to Install Cuckoo before being able to use it in the first place, a simple modify-and-test development
sequence doesn’t work out-of-the-box as it used to do.
Is it possible to somehow debug such application?
Later in the (unfinished) docs there is a mention of debugging web interface which I am not sure would get me to the code I need to debug and it also requires PyCharm Professional. I am using PyCharm Community but I am willing to use any other free IDE or debugger.
It will be difficult to debug if you are working only with compiled .pyo and .pyc files, which is all that you might have from an installed application. It is possible that the install includes some source files. You will have to figure out where it is installed, which is platform specific. Since many projects (including cuckoo) are open source, I would recommend cloning the Github repository and debugging from that.
I would like to upgrade my Python version from 2.6.6 to the newest version of 3.6.5 on a server. However, there is a program that has been running for multiple days and will be running for a week or more.
Will my program continue to run until it finishes if I upgrade my Python version?
P.S.: I ran my Python program with the screen command so I wouldn't have to stop it to be able to upgrade my Python version.
Yes, your Python 2 script should continue to run while you install/upgrade Python 3, since Python 2 and 3 are different programs.
However, the installation will be be platform specific and there might be common dependencies, so I would test this on a different machine (e.g. local virtual machine) first.
Several aspects are to be considered here:
In general, a Python program that is running does not access the installed binaries of the interpreter anymore. Also all the modules already loaded will not be read again from disk during the execution time of this program. (In this aspect it differs from e. g. a Bash script which reads its script source while executing.) So you probably are safe in even removing the complete Python installation; of course that's not what you plan on and I wouldn't recommend it for other reasons, but even that should not influence your running Python program. If, however, your running Python program tries to import a module for the first time after the removal or upgrade of your Python installation, this might fail due to incompatibilities between the versions. Most programs do their imports only at the beginning of their runtime, though.
Python 2 and 3 are different languages. Scripts for the one need to be at least adjusted to run with the other. Adjustments aren't too hard to do, though. Most distributions have both installed side-by-side. Most of the time, Python2 is still standard (and accessed by typing python). python2 and python3 are then to make it explicit. This means that you probably should not remove Python 2 from your system, even if you plan on installing Python 3. There probably are scripts relying on the installation of Python 2.
There's a lot going on here that you've not explained/described, so it's hard to give you a definitive answer (such as the operating system you're using). This is also not exactly a programming question, so you might find that the question gets voted to close. That said...
It possible to run multiple versions of Python side by side. Because 2.x and 3.x are very different, it might be much better to have both versions installed. See this answer for some details on doing that:
Official multiple Python versions on the same machine?
It's likely that your code will need some modifications to run on Python 3, so you're going to have to redeploy a new version of your code at some point anyway.
I've spent countless hours trying to understand this and unfortunately I haven't gotten to an answer yet. Or at least I don't think I have.
First up I should say that I am a Java Developer. I've only recently started working with Python and the build-process is a bit...odd for me.
In my mind I write an application, I compile it to run and I package it into a .jar for other people to use. Either as a library or for end-users to execute and have fun with it. (ignoring stuff like maven or gradle...)
I wrote a little CLT in python that consists of ~6 files and I wanted to distribute it. From what I could find I was supposed to write a setup.py and I found some guides on how to do that but ... to be honest I'm not even sure what that did. I could get my source code bundled into a tar.gz with some other meta data or it would create some weird files that I don't know what to do with.
Then I found PyInstaller and it worked great to package everything into a binary. However I've run into some problems trying to create a Debian package and it has made me re-assess and question the fact that there doesn't seem to be something in Python (without having to use an external tool) that lets me package/ bundle/ whatever my application into a single file to be run.
And that's something I can't get my head around. I mean...before there were tools like PyInstaller and P2Exe and what not, how did people distribute their applications? Am I expected to write a C application, somehow include the python code in there and compile that? Sorry if this seems like a stupid question but I'm really asking. I've googled around so much and spent so much time on it and haven't found a satisfactory answer so I hope someone here can help me with this! Thanks.
If you package your Python code for pip, you can include some executable scripts that start your program. I don't know how the situation was 5 years ago when this question got asked, but nowadays pip is pretty much integrated with Python, to the point that there's a standard library module to bootstrap pip in case it's missing:
https://docs.python.org/3/library/ensurepip.html
The situation is different if you want to package an application for some other package manager, like Anaconda or the package managers of various Linux distributions, or as a Windows installer. Obviously, you'll have to create a separate package for each package manager or installation technique you want to support.
So I wrote a Python script which does some simple stuff. It was originally going to run on a Unix server but due to crappy network security settings which TPTB refuse to change, we need to run it on a Windows server instead. However, the administrators of said Windows server refuse to do anything helpful like install Python.
What are my options for running a Python script on Windows without Python?
Consideration 1:
Something like Py2Exe - I found this after a quick Google search and it seems promising. From what I can tell, it'll generate a bunch of files but we can just xcopy that directory to our Windows machine and it will be completely isolated and not have any external dependencies. Does anyone have any insight on how well this works? Obviously, it depends on my Python script but fortunately this script is quite simple and only uses built in Python libraries such as urllib2 and urlparse.
Consideration 2:
We can assume the Windows server has at least some version of the .NET Framework installed too, which brings IronPython to mind. I've never used this before, but I've always wanted to. From what I can tell, it will compile Python code into CLS compliant IL code which can be run natively under the .NET runtime. However, does this require additional .NET libraries to be installed on the server? Can I just bundle those DLLs with my program? Or, does it require I rewrite my Python script to call into .NET Framework specific classes instead of using things like urllib2 or urlparse?
Thanks!
PS - The ironic part: I actually barely know Python and I'm a .NET expert, but I wrote the script in Python because I was told it would run on a Unix server. Had I known we'd end up running this on a Windows server, I'd have written the thing in C# to begin with in about 1/10th of the time. Fail.
Will they let you copy executables onto the server at all? If so then you should be able to do a non-admin installation of Python or use Portable Python which can just be copied into a folder without any installation at all.
Nothing wrong with Py2exe, but it does mean you then have to build the script into a fresh executable each time you update it. Also Py2exe has a slightly longer startup time than a Python interpreter because it has to extract the Python dlls into a temporary folder each time it runs; that only matters of course if you run your script a lot.