Just learning how to use Meson and want to generate protobuf source/headers for multiple languages - C++, Python, Java, Javascript. C++ was simple enough using the generator function in my meson.build file:
project('MesonProtobufExample', 'cpp')
protoc = find_program('protoc', required : true)
deps = dependency('protobuf', required : true)
gen = generator(protoc, \
output : ['#BASENAME#.pb.cc', '#BASENAME#.pb.h'],
arguments : ['--proto_path=#CURRENT_SOURCE_DIR#', '--cpp_out=#BUILD_DIR#', '#INPUT#'])
generated = gen.process('MyExample.proto')
ex = executable('my_example', 'my_example.cpp', generated, dependencies : deps)
Which produces the MyExample.pb.cc and MyExample.pb.h files. I figured Python would be just as easy but I'm a bit stumped since there's no executable() step for my Python script since it doesn't need to be compiled. I noticed meson (and CMake it turns out) don't actually generate the protobuf files until you call executable() so I can't just skip this step or the MyExample_pb2.py file will not be generated. I have found no example for using meson/python/GPB together after several hours of searching. Shouldn't there be a simple way to 'link' the generated sources to a python file/module like the way CMake does?
protobuf_generate_python(PROTO_PY MyExample.proto)
# This command causes the protobuf python binding to be generated
add_custom_target(my_example.py ALL DEPENDS ${PROTO_PY})
You can use trick with custom_target() and "fake compiler" in the form of cp or cat tools (in -nix environments, of course, if you want to support Windows then you can use conditional find_program()). Here is the example with cp:
py_gen = generator( ... )
py_generated = gen.process('MyExample.proto')
py_proc = custom_target('py_proto',
command: [ 'cp', '#INPUT#', '#OUTPUT#' ],
input : py_generated,
output : 'MyExample_pb2.py',
build_by_default : true)
I added buid_by_default flag assuming that you need to generate it as a part of standard build process (of course, enabling this target can be conditional too).
It's the first time i have come across a recipe file written in python and it's giving me an error. The error is:
../meta-intel/recipes-rt/images/core-image-rt.bb: Error executing a python function in <code>:
This is a recipe which is coming from the meta-intel branch "[master] intel-vaapi-driver: 2.1.0 -> 2.2.0".
My poky version is" [morty] documentation: Updated manual revision table for 2.2.4 release date.
My BITBAKE version is: "BitBake Build Tool Core version 1.32.0"
The contents of core-image-rt.bb are:
require recipes-core/images/core-image-minimal.bb
# Skip processing of this recipe if linux-intel-rt is not explicitly specified as the
# PREFERRED_PROVIDER for virtual/kernel. This avoids errors when trying
# to build multiple virtual/kernel providers.
python () {
if d.getVar("PREFERRED_PROVIDER_virtual/kernel") != "linux-intel-rt":
raise bb.parse.SkipPackage("Set PREFERRED_PROVIDER_virtual/kernel to linux-intel-rt to enable it")
}
DESCRIPTION = "A small image just capable of allowing a device to boot plus a \
real-time test suite and tools appropriate for real-time use."
DEPENDS += "linux-intel-rt"
IMAGE_INSTALL += "rt-tests hwlatdetect"
LICENSE = "MIT"
If you need any additional information please let me know and i'll try and supply it.
I can normally build images on my ubuntu machine but don't believe have ever had to build an image in which the recipes were written in python
You are using incompatible API of using g.getVar method. In morty release as the last one with old way of using second parameter, there is still need to provide boolean parameter:
...
if d.getVar("PREFERRED_PROVIDER_virtual/kernel", True) != "linux-intel-rt":
...
Please take a look at one of the commit, that remove this in next releases.
I am trying to do Incomplete Cholesky Decomposition in Python, but no direct Python package I can find.
Since the most available codes I can find online are written in Matlab, I want to take a detour by
compiling the matlab code to a shared library (I am using Mac OS and MATLAB_R2014a, so it should produce .dylib file)
load library in Python by using Ctypes
The following lists the detailed steps:
0. Download Matlab Source Code
The code can be downloaded from F. Bach's webpage link to zip file, which contains the following files:
panc:csi-1.0 panc25$ ls
center.m csi.dll csi.mexglx csi_gaussian.dll csi_gaussian.mexglx readme.txt
csi.c csi.m csi_gaussian.c csi_gaussian.m demo_csi.m sqdist.m
1. Compiling the matlab code to a shared library
Then by following this post, I run the command:
mcc -v -W cpplib:libcsi -T link:lib csi
After around a minute, the terminal prints MEX completed successfully and in my folder there are
panc:csi-1.0 panc25$ ls
center.m csi.m csi_gaussian.dll demo_csi.m libcsi.exports readme.txt
csi.c csi.mexglx csi_gaussian.m libcsi.cpp libcsi.h sqdist.m
csi.dll csi_gaussian.c csi_gaussian.mexglx libcsi.dylib mccExcludedFiles.log
where libcsi.dylib is the shared library I want.
2. Loading library in Python
Then I open IPython and try to load the library:
In [1]: import ctypes
In [2]: ctypes.C
ctypes.CDLL ctypes.CFUNCTYPE
In [2]: ctypes.CDLL('libcsi.dylib')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-2-b6d0c1a91651> in <module>()
----> 1 ctypes.CDLL('libcsi.dylib')
/Users/panc25/anaconda/lib/python2.7/ctypes/__init__.pyc in __init__(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle
OSError: dlopen(libcsi.dylib, 6): Library not loaded: #rpath/libmwmclmcrrt.8.3.dylib
Referenced from: /Users/panc25/Downloads/csi-1.0/libcsi.dylib
Reason: image not found
This problem persists even after I replace file name in ctypes.CDLL('libcsi.dylib') with the full path.
So I am confused. The shared library is there, but why Python says "image not found"?
BTW
SInce the source code also provide C implementation through mex.h, I also tried to first create a .mex file, then compile the .mex to a shared library as follows:
panc:csi-1.0 panc25$ mex csi.c
which created the csi.mexmaci64 file. Then according to this link, I called:
panc:csi-1.0 panc25$ mcc -B csharedlib:csi2 csi.mexmaci64
which produced csi2.dylib file.
But when I tried to load it in Python, I had the same error.
Could anyone let me know what is wrong?
I would avoid Matlab altogether, and instead use the Incomplete Cholesky Decomposition available in PyMC2:
from pymc.gp.incomplete_chol import ichol_full
The f2py wrapped Fortran code, that was actually adapted from a MEX file, can be found here. So you could use this independently of PyMC2 if need be.
If you are interested, you could also propose to add this function to scipy (see this githib issue ).
Using Python 2.7.9 with GDAL 1.11.1, with miniconda for package management --
Performing this a simple reprojection of a coordinate point causes the error described below.
I am relatively new to GDAL, so I checked to see if the code from the Python GDAL/OGR 1.0 Cookbook produces the same issue, and it does:
from osgeo import ogr
from osgeo import osr
source = osr.SpatialReference()
source.ImportFromEPSG(2927)
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
transform = osr.CoordinateTransformation(source, target)
point = ogr.CreateGeometryFromWkt("POINT (1120351.57 741921.42)")
point.Transform(transform)
print point.ExportToWkt()
This is the error:
/opt/miniconda/envs/pygeo/lib/python2.7/site-packages/osgeo/ogr.pyc in Transform(self, *args)
4880 OGRERR_NONE on success or an error code.
4881 """
-> 4882 return _ogr.Geometry_Transform(self, *args)
4883
4884 def GetSpatialReference(self, *args):
TypeError: in method 'Geometry_Transform', argument 2 of type 'OSRCoordinateTransformationShadow *'
CoordinateTransform is a proxy for the C++ OSRCoordinateTransformationShadow class, generated by SWIG.
Per the source code for osgeo.ogr.Geometry (what Point is), the correct types were passed to the Transform method.
Best guess: Could this be caused by using a version of _ogr that is too old, and so the implementation of _ogr.Geometry_Transform(self, *args) is expecting a different?
_ogr is another SWIG-generated proxy, I'm guessing for the OGR class?
What everyone new to GDAL must learn: assign an error handler. (example: http://pcjericks.github.io/py-gdalogr-cookbook/gdal_general.html#install-gdal-ogr-error-handler)
With an error handler assigned, the output includes the explanation for the error. In this case, it was: "Unable to load PROJ.4 library (libproj.so), creation of OGRCoordinateTransformation failed."
Hopefully, imparting the knowledge of enabling GDAL error handling will help others who may stumble upon this very issue.
Similar information can be found on a rasterio FAQ and in unable to load "gcs.csv" file in gdal.
I encountered this problem when running GDAL transformations in my Anaconda3 QGIS environment. The problem is that the coordinate system informations were not loading through the GDAL_DATA environment variable.
To remedy, locate where the directory containing gcs.csv exists within your system (potentially ".../Library/share/gdal"). Add this to your environment prior to importing GDAL & other dependents.
import os
os.environ['GDAL_DATA'] = r'/path/to/dir/'
With help of the answer of jeremy that GDAL fails to load its information, I just edited the code to specify directly the PROJ.4 parameters from the EPSG webside and it runs
#target.ImportFromEPSG(4326)
target.ImportFromProj4('+proj=longlat +datum=WGS84 +no_defs')
I've made simple code editor using wxPython. File size (python files) is 1.3 KB. But when I create executable using PyInstaller, I get 30 MB file! Is there a way to decrease file size? Btw, I am not importing whole wx library, only components I need (ex from wx import Frame).
Using Linux, Fedora 18 64bit.
wxPython is a big library so when you create an executable, they tend to end up being between 20 and 30 MB. Also note that Python itself is kind of bulky because Python is an interpreted language. So you are also including the Python interpreter when you create the exe.
With py2exe, I have gotten the executable below 10 MB, but it's a pain and doesn't work for all projects. It really depends on what else you are using. You can read about my adventures with py2exe here.
The other way to get it smaller is to use a compression program. That sometimes works and sometimes doesn't.
You can also tell most of these binary creation tools to exclude items. You can try that too.
I shipped a fairly simple wxPython app and it ended up being ~9.8MB.
If you utilize the ArchiveViewer.py script that's part of PyInstaller you can determine what's taking up so much space.
This was with python 2.7.5, without UPX, and excluding these modules:
excludesPassedToAnalysis = ['ssl',
'_ssl',
# coverage uses _socket. :(
#'_socket',
'select',
'pywin',
'unittest',
'win32ui',
'bz2',
'doctest',
'os2emxpath',
'servicemanager',
'xml.parsers.expat',
'sitecustomize',
'tarflie',
'email',
'urllib',
'urllib2',
# This exclude isn't optional in order to get pubsub working
# correctly in wxPython 2.9.3 or later.
'wx.lib.pubsub.autosetuppubsubv1']
# These are removed from a.pure after the Analysis object is created.
excludeEncodings = \
['encodings.base_64_codec',
'encodings.big5',
'encodings.big5hkscs',
'encodings.bz2_codec',
'encodings.cp037',
'encodings.cp1006',
'encodings.cp1026',
'encodings.cp1140',
'encodings.cp1258',
'encodings.cp424',
'encodings.cp437',
'encodings.cp500',
'encodings.cp720',
'encodings.cp737',
'encodings.cp775',
'encodings.cp850',
'encodings.cp852',
'encodings.cp855',
'encodings.cp856',
'encodings.cp857',
'encodings.cp858',
'encodings.cp860',
'encodings.cp861',
'encodings.cp862',
'encodings.cp863',
'encodings.cp864',
'encodings.cp865',
'encodings.cp866',
'encodings.cp869',
'encodings.cp874',
'encodings.cp875',
'encodings.cp932',
'encodings.cp949',
'encodings.cp950',
'encodings.euc_jis_2004',
'encodings.euc_jisx0213',
'encodings.euc_jp',
'encodings.euc_kr',
'encodings.gb18030',
'encodings.gb2312',
'encodings.gbk',
'encodings.hex_codec',
'encodings.hp_roman8',
'encodings.hz',
'encodings.iso2022_jp',
'encodings.iso2022_jp_1',
'encodings.iso2022_jp_2',
'encodings.iso2022_jp_2004',
'encodings.iso2022_jp_3',
'encodings.iso2022_jp_ext',
'encodings.iso2022_kr',
'encodings.iso8859_10',
'encodings.iso8859_11',
'encodings.iso8859_13',
'encodings.iso8859_14',
'encodings.iso8859_15',
'encodings.iso8859_16',
'encodings.iso8859_2',
'encodings.iso8859_3',
'encodings.iso8859_4',
'encodings.iso8859_5',
'encodings.iso8859_6',
'encodings.iso8859_7',
'encodings.iso8859_8',
'encodings.iso8859_9',
'encodings.johab',
'encodings.koi8_r',
'encodings.koi8_u',
'encodings.mac_arabic',
'encodings.mac_centeuro',
'encodings.mac_croatian',
'encodings.mac_cyrillic',
'encodings.mac_farsi',
'encodings.mac_greek',
'encodings.mac_iceland',
'encodings.mac_latin2',
'encodings.mac_roman',
'encodings.mac_romanian',
'encodings.mac_turkish',
'encodings.mbcs',
'encodings.palmos',
'encodings.ptcp154',
'encodings.quopri_codec',
'encodings.raw_unicode_escape',
'encodings.rot_13',
'encodings.shift_jis',
'encodings.shift_jis_2004',
'encodings.shift_jisx0213',
'encodings.string_escape',
'encodings.tis_620',
'encodings.undefined',
'encodings.utf_32',
'encodings.utf_32_be',
'encodings.utf_32_le',
'encodings.utf_7',
'encodings.uu_codec',
'encodings.zlib_codec',]