We have a python library that needs to be shared among many projects and we're trying to find a way to organize and link the shared library to the specific projects that want to use it.
It also needs to work without visual studio, meaning that if the whole project is moved to some different machine, it will still work and use the "shared library", which means that the linked library needs to be placed statically in every project that use it (and of course each time it's updated, the library directory will be updated in each project)
Is there anyway it can be done?
The directory structure looks like this:
Project1
main.py <--- (One of the projects that uses the library)
...
Libs
PyLib <--- (This is the shared library)
__init__.py
ps_lib.py
another.py
CWinLib
CNixLib
Some ways that I've tested are:
Working with linked files - The problem is that it doesn't copy the whole package to the project (which means that it doesn't work outside of visual studio)
Adding a search path - The same problem as before, doesn't work outside of visual studio
Using sys.path.append - It means that we'll need to copy the exact directory structure that's in place and that is something I want to avoid
Is there another way to solve this?
Related
I'm working on a Python package (built as a wheel), which requires some external files – i.e. ones that aren't in my package/repository. At runtime, it currently gets the path to these from a config file.
I'd like to instead bundle these files inside the wheel itself, so the package is all you need to install. Normally you'd use package_data to do this kind of thing, but I don't think that works for files outside of the package tree. I've wondered about making a build script, which first copies these into a local temporary directory. Would that work, or is there a more elegant way to do this?
I have a pretty specific problem regarding the my python setup. I need to reference python libraries made of .py files which are on my HD, but under a peculiar form of cloud-based version control. It is not possible to move those files elsewhere and I cannot add them to a python solution from VS(using VS2017). Basically, these are historically grown .py files right next to each other which reference each other. I would like to use VS2017 to work on and execute these python files and be able to reference the "neighboring" files without including them in a solution.
When I add these files to a python solution test.sln and adapt the search paths, everything works perfectly fine. I can reference anything, intellisense works, all good.
The modifications to the search paths are, as far as I can see, exclusive to test.sln. I added the source directories to the PYTHONPATH environment variable and disabled the "ignore global paths" option in VS, but still, referencing the .py files with each other without adding them to a solution does not work.
I can't find solution-independent reference search paths for VS, which would solve my problem. Is there a way to add default search paths for VS, or something like that?
# references:
import os
# works
import numpy as np
# works
import custom_file
# throws modulenotfound error
# do_stuff...
Example above.
Thank you for your help.
Just add sys.path.extend([path/to/custom_file_dir]) before trying to import the custom_file.
I am working with the 3.6.4 source release of Python. I have no trouble building it with Visual Studio as a dynamic library (/MDd) I can link the Python .dll to my own code and verify its operation.
But when I build it (and my code) with (/MTd) it soon runs off the rails when I try to open a file with a Python program. A Debug assertion fails in read.cpp ("Expression: _osfile(fh) & FOPEN"). What I believe is happening is the Python .dll is linking with improper system libraries. What I can't figure out is how to get it to link with the correct ones (static libraries).
This is what I needed to do to build and use python statically embedded in another application.
To build the static python library (e.g., python36_d.lib, python36.lib)
Convert ALL projects in the python solution (pcbuild.sln) to static. This is about 40 projects, so it may take awhile. This includes setting library products to be build as 'static lib', and setting all /MD and /MDd build options to /MT and /MTd.
For at least the pythoncore project alter the Preprocess define to be Py_NO_ENABLE_SHARED. This tells the project it will be looking for calls from static libraries.
By hook or crook, find yourself a pyconfig.h file and put it in the Include area of your Python build. It is unclear how this file is built from Windows tools, but one seems to be able to snag one from other sources and it works ok. One could probably grab the pyconfig.h from the Pre-compiled version of the code you are building. [By the way, the Python I built was 3.6.5 and was built with Windows 2015, update 3.]
Hopefully, this should enable you to build both python36.lib and python36_d.lib. Now you need to make changes to your application project(s) to enable it to link with the python library. You need to do this:
Add the Python Include directory to the General->Include Directories list.
Add the Python Library directories to the General->Library Directories lists.
This will be ..\PCBuild\win32 and ..\PCBuild\amd64.
Add the define Py_NO_ENABLE_SHARED to the C/C++ -> Preprocessor area.
For Linker->input add (for releases) python36.lib;shlwapi.lib;version.lib and (for debugs) python36_d.lib;shlwapi.lib;version.lib.
And that should be it. It should run and work. But one more thing. In order to be able to function, the executable needs to access the Lib directory of the python build. So a copy of that needs to be moved to wherever the executable (containing the embedded python) resides. Or you can add the Lib area to the execution PATH for windows. That should work as well.
That's about all of it.
Short version
Is it possible to build a SCons environment before the SConstruct script exits?
Long version
I'm porting some software from Windows to Linux. On Windows, it builds in Visual Studio 2013, using MSVC++ and Intel Fortran. On Linux, we're building it with g++ and gfortran.
I've written a Python script that reads a Visual Studio project file (either .vcxproj for the C++ code or .vfproj for the Fortran) and executes the relevant SCons builders to create the build. My SConstruct file then basically looks like this:
def convertVSProjectFile(filename):
...
projects = [ 'Source/Proj1/Proj1.vcxproj',
'Source/Proj2/Proj2.vcxproj',
'Source/Proj3/Proj3.vfproj',
...
];
for p in projects:
convertVSProjectFile(filename)
In time this will be reworked to interpret the .sln file rather than listing the projects manually.
For the C++ code, this works fine. It's a problem for the Fortran code, though. The problem comes up where files in two separate projects refer to the same Fortran module. The Fortran scanner spots this and makes the module's source file a dependency of both targets. However, the FORTRANMODPATH construction variable is set differently for the two targets. SCons warns that the same target is built twice with the same builder, but then seems to just pick one of them more or less at random, making it hard to predict where to .mod file will end up.
I can think of a few ways of fixing this:
- Construct each environment separately, build it, then move on to the next one. But I don't know if there's a way of doing this.
- Set the FORTRANMODPATH for each object file rather than each project. Then the .mod file can go in the object folder for the source file instead of all the .mod files for a project going in the same folder. But I can't spot a way of doing this either. Could I achieve this by creating a new Environment for every source file?
- Anything else anyone can come up with.
It is possible to specify the environment variables for each target
objs += env.Object(target=..., source=..., FORTRANMODPATH=...)
SCons will see the second use has different FORTRANMODPATH and should rebuild it as necessary.
it's a couple of months I've started development in Python - having myself a C# and Java background.
I'm currently working on 2 different python/appengine applications, and as often happens in those cases, both application share common code - so I would like to refactor and move the common/generic code into a shared place.
In either Java or C# I'd just create a new library project, move the code into the new project and add a reference to the library from the main projects.
I tried the same in Python, but I am unable to make it work.
I am using Eclipse with Pydev plugin.
I've created a new Pydev project, moved the code, and attempted to:
reference the library project from the main projects (using Project Properties -> Project References)
add the library src folder folder into the main projects (in this case I have an error - I presume it's not possible to leave the project boundaries when adding an existing source folder)
add as external library (pretty much the same as google libraries are defined, using Properties -> External libraries)
Import as link (from Import -> File System and enabling "Create links in workspace")
In all cases I am able to reference the library code while developing, but when I start debugging, the appengine development server throws an exception because it can't find what I have moved into a separate library project.
Of course I've searched for a solution a lot, but it looks like nobody has experienced the same problem - or maybe nobody doesn't need to do the same :)
The closest solution I've been able to find is to add an ant script to zip the library sources and copy in the target project - but this way debugging is a pain, as I am unable to step into the library code.
Any suggestion?
Needless to say, the proposed solution must take into account that the library code has to be included in the upload process to appengine...
Thanks
The dev_appserver and the production environment don't have any concept of projects or libraries, so you need to structure your app so that all the necessary libraries are under the application's root. The easiest way to do this, usually, is to symlink them in as subdirectories, or worst-case, to copy them (or, using version control, make them sub-repositories).
How that maps to operations in your IDE depends on the IDE, but in general, it's probably easiest to get the app structured as you need it on disk, and work backwards from that to get your IDE set up how you like it.