Develop Reference

Python setuptools_scm get version from git tags

I am using project.toml file to package my module, I want to extract the version from git tag using setuptools_scm module.
When I run python setup.p y --version command it gives this output 0.0.1.post1.dev0. How will I get only 0.0.1 value and omit the .post.dev0 value?
Here is project.toml file settings:
[build-system]
requires = ["setuptools>=46.1.0", "setuptools_scm[toml]>=5"]
build-backend = "setuptools.build_meta"
[tool.setuptools_scm]
version_scheme = "no-guess-dev"
local_scheme="no-local-version"
write_to = "src/showme/version.py"
git_describe_command = "git describe --dirty --tags --long --match v* --first-parent"
[tool.setuptools.dynamic]
version = {attr = "showme.__version__"}
output:
python setup.py --version
setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
warnings.warn(msg, _BetaConfiguration)
0.0.1.post1.dev0
Thanks

setuptools_scm out-of-the-box generates development and post-release versions.
To generate a release version like 0.0.1, you can pass a callable into use_scm_version:
# content of setup.py
def myversion():
from setuptools_scm.version import SEMVER_MINOR, guess_next_simple_semver, release_branch_semver_version
def my_release_branch_semver_version(version):
v = release_branch_semver_version(version)
if v == version.format_next_version(guess_next_simple_semver, retain=SEMVER_MINOR):
return version.format_next_version(guess_next_simple_semver, fmt="{guessed}", retain=SEMVER_MINOR)
return v
return {
'version_scheme': my_release_branch_semver_version,
'local_scheme': 'no-local-version',
}
setup(use_scm_version=myversion)
Reference: https://github.com/pypa/setuptools_scm#importing-in-setuppy

Python calling q# file on IONQ QPU results in error about a System.Text.Json, Version=5.0.0.0 file not being found

Attempting to learn how to call q# from Python code and have it run for real on the IONQ QPU as it does (or appears to do) using VS and >dotnet run of the raw q# code. I followed the guides and workshop.
Python code:
import qsharp
import qsharp.azure
qsharp.projects.add("****path to *******/TestIONQ.csproj")
from TestIONQ import GetRandomResult
print(f"Simulated Result: {GetRandomResult.simulate()}")
print("------------------------------------------------")
qsharp.azure.connect(
subscription = "****************************",
resourceGroup = "**************",
workspace = "************",
location = "******* US")
qsharp.azure.target("ionq.qpu")
result = qsharp.azure.execute(GetRandomResult, jobName="Generate random bit")
print(f" Final result from IONQ - QPU: {result}")
q# code:
namespace TestIONQ {
open Microsoft.Quantum.Canon;
open Microsoft.Quantum.Intrinsic;
//#EntryPoint()
operation GetRandomResult() : Result {
use q = Qubit();
H(q);
return M(q);
}
}
and the .csproj file:
<Project Sdk="Microsoft.Quantum.Sdk/0.16.2104138035">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<ExecutionTarget>ionq.qpu</ExecutionTarget>
<IQSharpLoadAutomatically>true</IQSharpLoadAutomatically>
</PropertyGroup>
</Project>
The results of running the above Python code in Anaconda qsharp-env environment (Python 3.7.10) are as follows:
Simulated Result: 0
------------------------------------------------
Connected to Azure Quantum workspace ####### in location #####us.
Loading package Microsoft.Quantum.Providers.IonQ and dependencies...
Active target is now ionq.qpu
Submitting TestIONQ.GetRandomResult to target ionq.qpu...
Failed to submit Q# operation TestIONQ.GetRandomResult for execution.
Could not load file or assembly 'System.Text.Json, Version=5.0.0.0, Culture=neutral, PublicKeyToken=####token#####'. The system cannot find the file specified.
Obviously, no problem connecting to Azure and the Workspace. In fact I can run the container-ship optimization example no problem from Python. This also works fine for the first half of the Python code when .simulate() is invoked.
Next, when I try to bypass the IONQ QPU and use its own simulator by changing this one line:
qsharp.azure.target("ionq.simulator")
The resulting error is the same and the results are as follows:
Simulated Result: 1
------------------------------------------------
Connected to Azure Quantum workspace ######## in location #######.
Loading package Microsoft.Quantum.Providers.IonQ and dependencies...
Active target is now ionq.simulator
Submitting TestIONQ.GetRandomResult to target ionq.simulator...
Failed to submit Q# operation TestIONQ.GetRandomResult for execution.
Could not load file or assembly 'System.Text.Json, Version=5.0.0.0, Culture=neutral, PublicKeyToken='....token......'. The system cannot find the file specified.
Traceback (most recent call last):
File "ionq_sim_remote.py", line 18, in <module>
result = qsharp.azure.execute(GetRandomResult, jobName="Generate random bit")
File "F:\Python38\miniconda\envs\qsharp-env\lib\site-packages\qsharp\azure.py", line 137, in execute
if "error_code" in result: raise AzureError(result)
qsharp.azure.AzureError: {'error_code': 1010, 'error_name': 'JobSubmissionFailed', 'error_description': 'Failed to submit the job to the Azure Quantum workspace.'}
This runs very easily on Azure using the q# code snippet within Visual Studio at the command line using a variant of what was shown during the workshop
az quantum execute --target-id ionq.qpu --job-name IONQ_test --resource-group ***rg name*** --workspace-name ***ws name*** --location **** -o table
and indeed this appears to have run on the actual QPU hardware as compared to the simulator (which gives the exact 0.5/0.5 result).
Result Frequency
-------- ----------
0 0.49800000
1 0.50200000
But then calling that same q# code from Python - including the same .csproj file seems to throw this JSON file error - even with the qsharp-env loaded into Anaconda. I apologize if it is something silly that I have done- trying to learn here.
By the way, this works great as a way around the problem with no Anaconda environment required or anything special:
Python:
import os
os.system(f'powershell.exe az quantum execute --target-id ionq.qpu --job-name Pytest --resource-group **** --workspace-name **** --location **** -o table ')
And the result was definitely run on the actual hardware (took a good while):
Result Frequency
-------- ----------- -------------------------
0 0.53200000 ▐███████████ |
1 0.46800000 ▐█████████ |

#Joab.Ai, thank you for posting this issue! We've identified this to be specific to the latest version of qsharp (0.16.2104.138035).
While we are looking into a fix, a workaround will be to downgrade your qsharp package version:
Edit: we have fixed this issue in our latest release! Update to the latest version with this command:
conda install -c quantum-engineering qsharp=0.16.2105.140472
or simply run:
conda update -c quantum-engineering qsharp

Using a custom docker with Azure ML

I'm following the guidelines (https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-environments) to use a custom docker file on Azure. My script to create the environment looks like this:
from azureml.core.environment import Environment
myenv = Environment(name = "myenv")
myenv.docker.enabled = True
dockerfile = r"""
FROM mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04
RUN apt-get update && apt-get install -y libgl1-mesa-glx
RUN echo "Hello from custom container!"
"""
myenv.docker.base_image = None
myenv.docker.base_dockerfile = dockerfile
Upon execution, this is totally ignored and libgl1 is not installed. Any ideas why?
EDIT: Here's the rest of my code:
est = Estimator(
source_directory = '.',
script_params = script_params,
use_gpu = True,
compute_target = 'gpu-cluster-1',
pip_packages = ['scipy==1.1.0', 'torch==1.5.1'],
entry_script = 'AzureEntry.py',
)
run = exp.submit(config = est)
run.wait_for_completion(show_output=True)
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-environments

Have no issues installing the lib. First, please dump your dockerfile content into a file, easier to maintain and read ;)
e = Environment("custom")
e.docker.base_dockerfile = "path/to/your/dockerfile"
will set the content of the file into a string prop.
e.register(ws).build(ws).wait_for_completion()
step 2/16 will have your apt update and libgl1 install
Note, that should work with >=1.7 SDK

This should work :
from azureml.core import Workspace
from azureml.core.environment import Environment
from azureml.train.estimator import Estimator
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core import Experiment
ws = Workspace (...)
exp = Experiment(ws, 'test-so-exp')
myenv = Environment(name = "myenv")
myenv.docker.enabled = True
dockerfile = r"""
FROM mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04
RUN apt-get update && apt-get install -y libgl1-mesa-glx
RUN echo "Hello from custom container!"
"""
myenv.docker.base_image = None
myenv.docker.base_dockerfile = dockerfile
## You need to instead put your packages in the Environment definition instead...
## see below for some changes too
myenv.python.conda_dependencies = CondaDependencies.create(pip_packages = ['scipy==1.1.0', 'torch==1.5.1'])
Finally you can build your estimator a bit differently :
est = Estimator(
source_directory = '.',
# script_params = script_params,
# use_gpu = True,
compute_target = 'gpu-cluster-1',
# pip_packages = ['scipy==1.1.0', 'torch==1.5.1'],
entry_script = 'AzureEntry.py',
environment_definition=myenv
)
And submit it :
run = exp.submit(config = est)
run.wait_for_completion(show_output=True)
Let us know if that works.

Totally understandable why you're struggling -- others have also expressed a need for more information.
perhaps base_dockerfile needs to be a text file (with the contents inside) and not a string? I'll ask the environments PM to learn more specifically how this works
another option would be to lever Azure Container Instance (ACI). An ACI is created automatically when spinning up an Azure ML workspace. See this GitHub issue for more info on that.

For more information about using Docker in environments, see the article `Enable
Docker https://learn.microsoft.com/azure/machine-learning/how-to-use-environments#enable-docker
The following example shows how to load docker steps as a string.
from azureml.core import Environment
myenv = Environment(name="myenv")
# Creates the environment inside a Docker container.
myenv.docker.enabled = True
# Specify docker steps as a string.
dockerfile = r'''
FROM mcr.microsoft.com/azureml/intelmpi2018.3-ubuntu16.04
RUN echo "Hello from custom container!"
'''
# Alternatively, load from a file.
#with open("dockerfiles/Dockerfile", "r") as f:
# dockerfile=f.read()
myenv.docker.base_dockerfile = dockerfile

I think it's that you're using an estimator. Estimators create their own environment, unless you set the environment_definition parameter, which I don't see in your snippet. I'm looking at https://learn.microsoft.com/en-us/python/api/azureml-train-core/azureml.train.estimator.estimator?view=azure-ml-py.
Haven't tried it, but I think you can fix this by changing your code to:
est = Estimator(
source_directory = '.',
script_params = script_params,
use_gpu = True,
compute_target = 'gpu-cluster-1',
pip_packages = ['scipy==1.1.0', 'torch==1.5.1'],
entry_script = 'AzureEntry.py',
environment_definition = myenv
)
run = exp.submit(config = est)
run.wait_for_completion(show_output=True)
You might also have to move use_gpu setting into the environment definition, as the SDK page I linked above says that the environment will take precedence over this and a couple other estimator parameters.

How to run pyleus on Storm

I'm a learning on Storm, I have installed zookeeper, storm, python and pyleus. the first step, I copy python script from pyleus web as a sample (https://github.com/Yelp/pyleus/tree/aaa423864f953332202832b8fd8404e03d3d74e3 ) and try to run it in storm server, the sample include below 3 files:
pyleus_topology.yaml, dummy_spout.py and dummy_bolt.py
the 2 py file has been put into a folder namely "my_first_topology" but when I run the pyleus build command in my VMware server(CentOS64-bit), the command can run 20 second and then, I got below error:
[root#localhost bin]# pyleus build /root/Desktop/CRM_ETL-Project-Storm/my_first_topology/pyleus_topology.yaml
pyleus build: error: [VirtualenvError] Failed to execute Python
module: my_first_topology.dummy_spout. Error:
/tmp/tmpZMIXa3/resources/pyleus_venv/bin/python: No module named
my_first_topology
what I can do for it? any steps I missed?
the script for reference
1> pyleus_topology.yaml
name: my_first_topology
topology:
- spout:
name: my-first-spout
module: my_first_topology.dummy_spout
- bolt:
name: my-first-bolt
module: my_first_topology.dummy_bolt
groupings:
- shuffle_grouping: my-first-spout
2> dummy_spout.py
from pyleus.storm import Spout
class DummySpout(Spout):
OUTPUT_FIELDS = ['sentence', 'name']
def next_tuple(self):
self.emit(("This is a sentence.", "spout",))
if name == 'main':
DummySpout().run()
3> dummy_bolt.py
from pyleus.storm import SimpleBolt
class DummyBolt(SimpleBolt):
OUTPUT_FIELDS = ['sentence']
def process_tuple(self, tup):
sentence, name = tup.values
new_sentence = "{0} says, \"{1}\"".format(name, sentence)
self.emit((new_sentence,), anchors=[tup])
if name == 'main':
DummyBolt().run()

I think your problem is that you are missing an empty file named "__init.py__" in your inner my_first_topology folder. That file makes it a python module. Just create it and you should be set.

How do I use cx_freeze?

I've created my setup.py file as instructed but I don't actually.. understand what to do next. Typing "python setup.py build" into the command line just gets a syntax error.
So, what do I do?
setup.py:
from cx_Freeze import setup, Executable
setup(
name = "On Dijkstra's Algorithm",
version = "3.1",
description = "A Dijkstra's Algorithm help tool.",
exectuables = [Executable(script = "Main.py", base = "Win32GUI")])

Add import sys as the new topline
You misspelled "executables" on the last line.
Remove script = on last line.
The code should now look like:
import sys
from cx_Freeze import setup, Executable
setup(
name = "On Dijkstra's Algorithm",
version = "3.1",
description = "A Dijkstra's Algorithm help tool.",
executables = [Executable("Main.py", base = "Win32GUI")])
Use the command prompt (cmd) to run python setup.py build. (Run this command from the folder containing setup.py.) Notice the build parameter we added at the end of the script call.

I'm really not sure what you're doing to get that error, it looks like you're trying to run cx_Freeze on its own, without arguments. So here is a short step-by-step guide on how to do it in windows (Your screenshot looks rather like the windows command line, so I'm assuming that's your platform)
Write your setup.py file. Your script above looks correct so it should work, assuming that your script exists.
Open the command line (Start -> Run -> "cmd")
Go to the location of your setup.py file and run python setup.py build
Notes:
There may be a problem with the name of your script. "Main.py" contains upper case letters, which might cause confusion since windows' file names are not case sensitive, but python is. My approach is to always use lower case for scripts to avoid any conflicts.
Make sure that python is on your PATH (read http://docs.python.org/using/windows.html)1
Make sure are are looking at the new cx_Freeze documentation. Google often seems to bring up the old docs.

I ran into a similar issue. I solved it by setting the Executable options in a variable and then simply calling the variable. Below is a sample setup.py that I use:
from cx_Freeze import setup, Executable
import sys
productName = "ProductName"
if 'bdist_msi' in sys.argv:
sys.argv += ['--initial-target-dir', 'C:\InstallDir\\' + productName]
sys.argv += ['--install-script', 'install.py']
exe = Executable(
script="main.py",
base="Win32GUI",
targetName="Product.exe"
)
setup(
name="Product.exe",
version="1.0",
author="Me",
description="Copyright 2012",
executables=[exe],
scripts=[
'install.py'
]
)

You can change the setup.py code to this:
from cx_freeze import setup, Executable
setup( name = "foo",
version = "1.1",
description = "Description of the app here.",
executables = [Executable("foo.py")]
)
I am sure it will work. I have tried it on both windows 7 as well as ubuntu 12.04

find the cxfreeze script and run it. It will be in the same path as your other python helper scripts, such as pip.
cxfreeze Main.py --target-dir dist
read more at:
http://cx-freeze.readthedocs.org/en/latest/script.html#script

I usually put the calling setup.py command into .bat file to easy recall.
Here is simple code in COMPILE.BAT file:
python setup.py build
#ECHO:
#ECHO . : ` . * F I N I S H E D * . ` : .
#ECHO:
#Pause
And the setup.py is organized to easy customizable parameters that let you set icon, add importe module library:
APP_NAME = "Meme Studio"; ## < Your App's name
Python_File = "app.py"; ## < Main Python file to run
Icon_Path = "./res/iconApp48.ico"; ## < Icon
UseFile = ["LANGUAGE.TXT","THEME.TXT"];
UseAllFolder = True; ## Auto scan folder which is same level with Python_File and append to UseFile.
Import = ["infi","time","webbrowser", "cv2","numpy","PIL","tkinter","math","random","datetime","threading","pathlib","os","sys"]; ## < Your Imported modules (cv2,numpy,PIL,...)
Import+=["pkg_resources","xml","email","urllib","ctypes", "json","logging"]
################################### CX_FREEZE IGNITER ###################################
from os import walk
def dirFolder(folderPath="./"): return next(walk(folderPath), (None, None, []))[1]; # [ Folder ]
def dirFile(folderPath="./"): return next(walk(folderPath), (None, None, []))[2]; # [ File ]
if UseAllFolder: UseFile += dirFolder();
import sys, pkgutil;
from cx_Freeze import setup, Executable;
BasicPackages=["collections","encodings","importlib"] + Import;
def AllPackage(): return [i.name for i in list(pkgutil.iter_modules()) if i.ispkg]; # Return name of all package
#Z=AllPackage();Z.sort();print(Z);
#while True:pass;
def notFound(A,v): # Check if v outside A
try: A.index(v); return False;
except: return True;
build_msi_options = {
'add_to_path': False,
"upgrade_code": "{22a35bac-14af-4159-7e77-3afcc7e2ad2c}",
"target_name": APP_NAME,
"install_icon": Icon_Path,
'initial_target_dir': r'[ProgramFilesFolder]\%s\%s' % ("Picox", APP_NAME)
}
build_exe_options = {
"includes": BasicPackages,
"excludes": [i for i in AllPackage() if notFound(BasicPackages,i)],
"include_files":UseFile,
"zip_include_packages": ["encodings"] ##
}
setup( name = APP_NAME,
options = {"build_exe": build_exe_options},#"bdist_msi": build_msi_options},#,
executables = [Executable(
Python_File,
base='Win32GUI',#Win64GUI
icon=Icon_Path,
targetName=APP_NAME,
copyright="Copyright (C) 2900AD Muc",
)]
);
The modules library list in the code above is minimum for workable opencv + pillow + win32 application.
Example of my project file organize:
========== U P D A T E ==========
Although cx_Freeze is a good way to create setup file. It's really consume disk space if you build multiple different software project that use large library module like opencv, torch, ai... Sometimes, users download installer, and receive false positive virus alert about your .exe file after install.
Thus, you should consider use SFX archive (.exe) your app package and SFX python package separate to share between app project instead.
You can create .bat that launch .py file and then convert .bat file to .exe with microsoft IExpress.exe.
Next, you can change .exe icon to your own icon with Resource Hacker: http://www.angusj.com/resourcehacker/
And then, create SFX archive of your package with PeaZip: https://peazip.github.io/
Finally change the icon.
The Python Package can be pack to .exe and the PATH register can made with .bat that also convertable to .exe.
If you learn more about command in .bat file and make experiments with Resource Hacker & self extract ARC in PeaZip & IExpress, you can group both your app project, python package into one .exe file only. It'll auto install what it need on user machine. Although this way more complex and harder, but then you can custom many install experiences included create desktop shorcut, and install UI, and add to app & feature list, and uninstall ability, and portable app, and serial key require, and custom license agreement, and fast window run box, and many more,..... but the important features you get is non virus false positive block, reduce 200MB to many GB when user install many your python graphic applications.