Short version
Is it possible to build a SCons environment before the SConstruct script exits?
Long version
I'm porting some software from Windows to Linux. On Windows, it builds in Visual Studio 2013, using MSVC++ and Intel Fortran. On Linux, we're building it with g++ and gfortran.
I've written a Python script that reads a Visual Studio project file (either .vcxproj for the C++ code or .vfproj for the Fortran) and executes the relevant SCons builders to create the build. My SConstruct file then basically looks like this:
def convertVSProjectFile(filename):
...
projects = [ 'Source/Proj1/Proj1.vcxproj',
'Source/Proj2/Proj2.vcxproj',
'Source/Proj3/Proj3.vfproj',
...
];
for p in projects:
convertVSProjectFile(filename)
In time this will be reworked to interpret the .sln file rather than listing the projects manually.
For the C++ code, this works fine. It's a problem for the Fortran code, though. The problem comes up where files in two separate projects refer to the same Fortran module. The Fortran scanner spots this and makes the module's source file a dependency of both targets. However, the FORTRANMODPATH construction variable is set differently for the two targets. SCons warns that the same target is built twice with the same builder, but then seems to just pick one of them more or less at random, making it hard to predict where to .mod file will end up.
I can think of a few ways of fixing this:
- Construct each environment separately, build it, then move on to the next one. But I don't know if there's a way of doing this.
- Set the FORTRANMODPATH for each object file rather than each project. Then the .mod file can go in the object folder for the source file instead of all the .mod files for a project going in the same folder. But I can't spot a way of doing this either. Could I achieve this by creating a new Environment for every source file?
- Anything else anyone can come up with.
It is possible to specify the environment variables for each target
objs += env.Object(target=..., source=..., FORTRANMODPATH=...)
SCons will see the second use has different FORTRANMODPATH and should rebuild it as necessary.
Related
I've tried looking online and I'm honestly lost at this point.
I'm trying to find if there's a way to import python scripts and run them AFTER my Python program has been compiled.
For an example, let's say I have a main.py such that:
import modules.NewModule
a = NewModuleClass()
a.showYouWork()
then I compile main.py with pyinstaller so that my directory looks like:
main.exe
modules/NewModule.py
My end goal is to make a program that can dynamically read new Python files in a folder (this will be coded in) and use them (the part I'm struggling with). I know it's possible, since that's how add-ons work in Blender 3D but I've struggled for many hours to figure this out. I think I'm just bad at choosing the correct terms in Google.
Maybe I just need to convert all of the Python files in the modules directory to .pyc files? Then, how would I use them?
Also, if this is a duplicate on here (it probably is), please let me know. I couldn't find this issue on this site either.
You may find no detailed answer simply because there is no problem. PyInstaller does not really compile Python scripts into machine code executables. It just assembles then into a folder along with an embedded Python interpretor, or alternatively creates a compressed single file executable that will automatically uncompress itself at run time into a temporary folder containing that.
From then on, you have an almost standard Python environment, with normal .pyc file which can contain normal Python instructions like calls to importlib to dynamically load other Python modules. You have just to append the directory containing the modules to sys.path before importing them. An other possible caveat, is that pyinstaller only gets required modules and not a full Python installation, so you must either make sure that the dynamic modules do not rely on missing standard modules, or be prepared to face an ImportError.
I am working with the 3.6.4 source release of Python. I have no trouble building it with Visual Studio as a dynamic library (/MDd) I can link the Python .dll to my own code and verify its operation.
But when I build it (and my code) with (/MTd) it soon runs off the rails when I try to open a file with a Python program. A Debug assertion fails in read.cpp ("Expression: _osfile(fh) & FOPEN"). What I believe is happening is the Python .dll is linking with improper system libraries. What I can't figure out is how to get it to link with the correct ones (static libraries).
This is what I needed to do to build and use python statically embedded in another application.
To build the static python library (e.g., python36_d.lib, python36.lib)
Convert ALL projects in the python solution (pcbuild.sln) to static. This is about 40 projects, so it may take awhile. This includes setting library products to be build as 'static lib', and setting all /MD and /MDd build options to /MT and /MTd.
For at least the pythoncore project alter the Preprocess define to be Py_NO_ENABLE_SHARED. This tells the project it will be looking for calls from static libraries.
By hook or crook, find yourself a pyconfig.h file and put it in the Include area of your Python build. It is unclear how this file is built from Windows tools, but one seems to be able to snag one from other sources and it works ok. One could probably grab the pyconfig.h from the Pre-compiled version of the code you are building. [By the way, the Python I built was 3.6.5 and was built with Windows 2015, update 3.]
Hopefully, this should enable you to build both python36.lib and python36_d.lib. Now you need to make changes to your application project(s) to enable it to link with the python library. You need to do this:
Add the Python Include directory to the General->Include Directories list.
Add the Python Library directories to the General->Library Directories lists.
This will be ..\PCBuild\win32 and ..\PCBuild\amd64.
Add the define Py_NO_ENABLE_SHARED to the C/C++ -> Preprocessor area.
For Linker->input add (for releases) python36.lib;shlwapi.lib;version.lib and (for debugs) python36_d.lib;shlwapi.lib;version.lib.
And that should be it. It should run and work. But one more thing. In order to be able to function, the executable needs to access the Lib directory of the python build. So a copy of that needs to be moved to wherever the executable (containing the embedded python) resides. Or you can add the Lib area to the execution PATH for windows. That should work as well.
That's about all of it.
I'm trying to build a simple IDE that is web based in Python. For now, this IDE will support C only. I know it is possible to call the gcc with Python to compile and run a single C file. But what if I would like to compile and run multiple C files from a single project (i.e. linking .h files and .c files), is this possible? If yes, can you please tell me how?
Well assuming you want to handle all type of projects and their dependencies (which is not easy) the best way is to have a module that generates a Makefile for the project and use it to compile and solve all dependencies
We have a python library that needs to be shared among many projects and we're trying to find a way to organize and link the shared library to the specific projects that want to use it.
It also needs to work without visual studio, meaning that if the whole project is moved to some different machine, it will still work and use the "shared library", which means that the linked library needs to be placed statically in every project that use it (and of course each time it's updated, the library directory will be updated in each project)
Is there anyway it can be done?
The directory structure looks like this:
Project1
main.py <--- (One of the projects that uses the library)
...
Libs
PyLib <--- (This is the shared library)
__init__.py
ps_lib.py
another.py
CWinLib
CNixLib
Some ways that I've tested are:
Working with linked files - The problem is that it doesn't copy the whole package to the project (which means that it doesn't work outside of visual studio)
Adding a search path - The same problem as before, doesn't work outside of visual studio
Using sys.path.append - It means that we'll need to copy the exact directory structure that's in place and that is something I want to avoid
Is there another way to solve this?
I don't even know the right way to put my question but i will try my best.
I downloaded (Python-3.4.2.tar) the source code of a python interpreter from www.python.org
I extracted the files(using 7-zip).
Now lets say i latter want to use the unziped/extracted fies to create an installer i.e put it in a form that i can double click and Python-3.4.2 will be installed in my computer.
i guess it is called creating a build distriution.
I know i can just download Python-3.4.2.exe from the site and install right away but i want to know how it goes from being source code to becoming something one can install.
Here's a starting point for compiling:
https://github.com/python/cpython/blob/2.7/PCbuild/readme.txt
I was able to compile 2.7.13 with:
cd PCBuild
cmd /c get_externals.bat
cmd /c build.bat -e --no-tkinter "/p:PlatformToolset=v100"
3.4 may require a later compiler.
Here's how to build an installer:
https://github.com/python/cpython/blob/2.7/Tools/msi/README.txt
For building the CPython python interpreter from source, you'll want to have a look at the instructions at https://docs.python.org/devguide/
These instructions probably also (somewhere) contain the steps to produce a Windows installer, which seems to be done with some tool called PCbuild.
Basically, I think what you are asking is how you can distribute the Python runtime along with your program. The process is pretty simple. First, you may want to take a look at Python's distutils. Secondly, you will need to distribute the Python runtime. Python is currently released within installation binaries for pretty much every major operating system. You have the option of compiling target system(s) yourself in order to make Python silently install.
You will then need to decide where you want the files to go. Most people don't like this sort of behavior, so I recommend putting the files in your programs directory. Shortcuts, system variables, et cetera will need to be looked into.
As another option you can consider porting your script(s) to Jython, as most people tend to have at least the Java runtime. In the process of porting your code there is a way of constructing Java .class files with Jython when coded a certain way. They can then be placed into an executable Jar file. Easy peasy, but I haven't tried this myself, to be honest.
If you want to stick with pure Python, the last hurtle is putting everything in one file. There are plenty of tools for that. I would try Github for starters. For bonus points you can always code your own self-extracting binary.
Post Script: With the title changed, I think you are looking for Creating Build Distributions. It is the second link within the Google results for the query "how to compile python installer". Further down the same page I found this gem.