I'm a learning on Storm, I have installed zookeeper, storm, python and pyleus. the first step, I copy python script from pyleus web as a sample (https://github.com/Yelp/pyleus/tree/aaa423864f953332202832b8fd8404e03d3d74e3 ) and try to run it in storm server, the sample include below 3 files:
pyleus_topology.yaml, dummy_spout.py and dummy_bolt.py
the 2 py file has been put into a folder namely "my_first_topology" but when I run the pyleus build command in my VMware server(CentOS64-bit), the command can run 20 second and then, I got below error:
[root#localhost bin]# pyleus build /root/Desktop/CRM_ETL-Project-Storm/my_first_topology/pyleus_topology.yaml
pyleus build: error: [VirtualenvError] Failed to execute Python
module: my_first_topology.dummy_spout. Error:
/tmp/tmpZMIXa3/resources/pyleus_venv/bin/python: No module named
my_first_topology
what I can do for it? any steps I missed?
the script for reference
1> pyleus_topology.yaml
name: my_first_topology
topology:
- spout:
name: my-first-spout
module: my_first_topology.dummy_spout
- bolt:
name: my-first-bolt
module: my_first_topology.dummy_bolt
groupings:
- shuffle_grouping: my-first-spout
2> dummy_spout.py
from pyleus.storm import Spout
class DummySpout(Spout):
OUTPUT_FIELDS = ['sentence', 'name']
def next_tuple(self):
self.emit(("This is a sentence.", "spout",))
if name == 'main':
DummySpout().run()
3> dummy_bolt.py
from pyleus.storm import SimpleBolt
class DummyBolt(SimpleBolt):
OUTPUT_FIELDS = ['sentence']
def process_tuple(self, tup):
sentence, name = tup.values
new_sentence = "{0} says, \"{1}\"".format(name, sentence)
self.emit((new_sentence,), anchors=[tup])
if name == 'main':
DummyBolt().run()
I think your problem is that you are missing an empty file named "__init.py__" in your inner my_first_topology folder. That file makes it a python module. Just create it and you should be set.
Related
I need to check the change of one parameter, if it has changed - I need to restart the script.
code:
import subprocess, os.path, time
from engine.db_manager import DbManager
DB = DbManager(os.path.abspath(os.path.join('../webserver/db.sqlite3')))
tmbot = None
telegram_config = DB.get_config('telegram')
old_telegram_token = ''
vkbot = None
vk_config = DB.get_config('vk')
old_vk_token = ''
while True:
telegram_config = DB.get_config('telegram')
if old_telegram_token != telegram_config['token']:
if vkbot != None:
tmbot.terminate()
tmbot = subprocess.Popen(['python', 'tm/tm_bot.py'])
old_telegram_token = telegram_config['token']
print('telegram token was updated')
vk_config = DB.get_config('vk')
if old_vk_token != vk_config['token']:
if vkbot != None:
vkbot.terminate()
vkbot = subprocess.Popen(['python', 'vk/vk_bot.py'])
old_vk_token = vk_config['token']
print('vk token was updated')
time.sleep(30)
I get errors:
enter image description here
While there might be subtle differences between unix and windows, the straight-up answer is: you can use PYTHONPATH environment variable to let python know where to look for libraries.
However, if you use venv, I'd recommend activating it first, or calling the relevant binary instead of setting the environment variable.
Consider this scenario: you have a venv at /tmp/so_demo/venv, and you try to run this file:
$ cat /tmp/so_demo/myscript.py
import requests
print("great success!")
Running the system python interpreter will not find the requests module, and will yield the following error:
$ python3 /tmp/so_demo/myscript.py
Traceback (most recent call last):
File "/tmp/so_demo/myscript.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
As I have installed requests in the venv, if I provide the path to python, it will know where to look:
$ PYTHONPATH='/tmp/so_demo/venv/lib/python3.8/site-packages/' python3 /tmp/so_demo/myscript.py
great success!
but using the binary inside the venv is better, as you don't need to be familiar with the internal venv directory paths, and is much more portable (notice that the path I provided earlier depicts the minor version):
$ /tmp/so_demo/venv/bin/python /tmp/so_demo/myscript.py
great success!
I am trying to use Bazel for my new project, and for some reason I can only get bazel 0.26.1. However, when I am trying to write a test case using py_test, it seems that bazel is always using Python 2 to test my program. Is there any way to prevent this behavior?
To reproduce:
file test_a.py:
# Works on Python 3
# SyntaxError on Python 2
print(print('Good'))
file WORKSPACE:
load("#bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(
name = "rules_python",
commit = "54d1cb35cd54318d59bf38e52df3e628c07d4bbc",
remote = "https://github.com/bazelbuild/rules_python.git",
)
file BUILD:
load("#rules_python//python:defs.bzl", "py_test")
py_test(
name = "test_a",
size = "small",
srcs = ["test_a.py"],
deps = [],
)
My shell looks like (... is a path in ~/.cache/)
$ bazel version | head -n 1
Build label: 0.26.1
$ bazel test test_a
//:test_a FAILED in 0.1s
.../test.log
INFO: Build completed, 1 test FAILED, 2 total actions
$ cat .../test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //:test_a
-----------------------------------------------------------------------------
File ".../test_a.py", line 1
print(print('Good'))
^
SyntaxError: invalid syntax
$
According to a note in the documentation of the python_version flag for py_test there is a bug (#4815) where the script may still invoke the wrong interpreter version at runtime.
The suggested workaround is to define a py_runtime rule using select() and
point to that py_runtime with the --python_top flag (see issue for more
details):
py_runtime(
name = "myruntime",
interpreter_path = select({
# Update paths as appropriate for your system.
"#bazel_tools//tools/python:PY2": "/usr/bin/python2",
"#bazel_tools//tools/python:PY3": "/usr/bin/python3",
}),
files = [],
)
> bazel test :test_a --python_top=//path/to:myruntime.
The issue appears to have been fixed in 0.27.0
I have been trying to run a code a found and I get this kind of error . I don't know how to deal with this because I am new to python and trying to understand the concept of TSP problem.
Any help would be appreciated
Code below
from libs.GeneticAlgorithm import GeneticAlgorithm
def main():
poland = Country()
poland.add([
City('Gorlice', (49.655299, 21.159769)),
City('Sosnowiec', (50.286263, 19.104078)),
City('Łódź', (51.760229, 19.457209)),
City('Wrocław', (51.108314, 17.037802)),
City('Poznań', (52.406376, 16.925167)),
City('Toruń', (53.013790, 18.598444)),
City('Zielona Góra', (51.935619, 15.506186)),
City('Szczecin', (53.428543, 14.552812)),
City('Rzeszów', (50.041187, 21.999121)),
City('Kraków', (50.049683, 19.944544)),
City('Olsztyn', (53.770226, 20.490189)),
City('Lublin', (51.245376, 22.568278))
])
print('Cities:', end=' ')
print(*(city for city in poland.cities), sep=', ')
ga = GeneticAlgorithm(100, mutation_rate=0.5, ptype=Route, args=(poland.cities,))
ga.run(seconds=10)
fittest = ga.alltime_best
best_fitness = fittest.fitness
print('Best route:', fittest)
print('Best fitness:', best_fitness)
print('Generations:', ga.generation)
if __name__ == '__main__':
main()
If you "found" this code (most probably on the internet?) then the one who posted this had a libs folder in his machine, with a GeneticAlgorhythm.py module in it, so you either find this GeneticAlgorhythm module, or you won't be able to run this code successfully.
When you see, in python from baz.bar import Foo, Python is going to look for the bar module into the baz folder, and import class Foo from it. So you need to have baz module on your PC, or else that error will appear
It comes from this repository: https://github.com/reconndev/Genetic-Algorithm-TSP. In order to get it running, simply clone or download the repository. Then, in the main (root) folder, run python TSP-Text.py.
python TSP-Text.py
Cities: Gorlice, Sosnowiec, Łódź, Wrocław, Poznań, Toruń, Zielona Góra, Szczecin, Rzeszów, Kraków, Olsztyn, Lublin
Best route: Toruń->Olsztyn->Łódź->Lublin->Rzeszów->Gorlice->Sosnowiec->Kraków->Wrocław->Zielona Góra->Szczecin->Poznań
Best fitness: 0.4219619417258425
Generations: 807
I have been searching since a couple of days for a solution without success.
We have a windows service build to copy some files from one location to another one.
So I build the code shown below with Python 3.7.
The full coding can be found on Github.
When I run the service using python all is working fine, I can install the service and also start the service.
This using commands:
Install the service:
python jis53_backup.py install
Run the service:
python jis53_backup.py start
When I now compile this code using pyinstaller with command:
pyinstaller -F --hidden-import=win32timezone jis53_backup.py
After the exe is created, I can install the service but when trying to start the service I get the error:
Error starting service: The service did not respond to the start or
control request in a timely fashion
I have gone through multiple posts on Stackoverflow and on Google related to this error however, without success. I don't have the option to install the python 3.7 programs on the PC's that would need to run this service. That's why we are trying to get a .exe build.
I have made sure to have the path updated according to the information that I found in the different questions.
Image of path definitions:
I also copied the pywintypes37.dll file.
From -> Python37\Lib\site-packages\pywin32_system32
To -> Python37\Lib\site-packages\win32
Does anyone have any other suggestions on how to get this working?
'''
Windows service to copy a file from one location to another
at a certain interval.
'''
import sys
import time
from distutils.dir_util import copy_tree
import servicemanager
import win32serviceutil
import win32service
from HelperModules.CheckFileExistance import check_folder_exists, create_folder
from HelperModules.ReadConfig import (check_config_file_exists,
create_config_file, read_config_file)
from ServiceBaseClass.SMWinService import SMWinservice
sys.path += ['filecopy_service/ServiceBaseClass',
'filecopy_service/HelperModules']
class Jis53Backup(SMWinservice):
_svc_name_ = "Jis53Backup"
_svc_display_name_ = "JIS53 backup copy"
_svc_description_ = "Service to copy files from server to local drive"
def start(self):
self.conf = read_config_file()
if not check_folder_exists(self.conf['dest']):
create_folder(self.conf['dest'])
self.isrunning = True
def stop(self):
self.isrunning = False
def main(self):
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
while self.isrunning:
# Copy the files from the server to a local folder
# TODO: build function to trigger only when a file is changed.
copy_tree(self.conf['origin'], self.conf['dest'], update=1)
time.sleep(30)
if __name__ == '__main__':
if sys.argv[1] == 'install':
if not check_config_file_exists():
create_config_file()
if len(sys.argv) == 1:
servicemanager.Initialize()
servicemanager.PrepareToHostSingle(Jis53Backup)
servicemanager.StartServiceCtrlDispatcher()
else:
win32serviceutil.HandleCommandLine(Jis53Backup)
I was also facing this issue after compiling using pyinstaller. For me, the issue was that I was using the paths to configs and logs file in dynamic way, for ex:
curr_path = os.path.dirname(os.path.abspath(__file__))
configs_path = os.path.join(curr_path, 'configs', 'app_config.json')
opc_configs_path = os.path.join(curr_path, 'configs', 'opc.json')
log_file_path = os.path.join(curr_path, 'logs', 'application.log')
This was working fine when I was starting the service using python service.py install/start. But after compiling it using pyinstaller, it always gave me error of not starting in timely fashion.
To resolve this, I made all the dynamic paths to static, for ex:
configs_path = 'C:\\Program Files (x86)\\ScantechOPC\\configs\\app_config.json'
opc_configs_path = 'C:\\Program Files (x86)\\ScantechOPC\\configs\\opc.json'
debug_file = 'C:\\Program Files (x86)\\ScantechOPC\\logs\\application.log'
After compiling via pyinstaller, it is now working fine without any error. Looks like when we do dynamic path, it doesn't get the actual path to files and thus it gives error.
Hope this solves your problem too. Thanks
i have basic python agi code with pyst lib as:
extensions.conf
[from-internal]
exten => _.,1,answer()
exten => _.,2,AGI(test.py)
i have test.py in /var/lib/asterisk/agi-bin as
#!/usr/bin/python
import sys
import os
from agi import AGI
def test_call(agi = None,text = ""):
agi.say_alpha(text, "#")
agi.hangup()
if __name__ == "__main__":
text = 'abcdefr'
agi = AGI()
test_call(agi,text)
and i have one file agi.py which is get from pyst lib.
when i try to call to this agi i get this issue
Executing [123#from-internal:1] Answer("SIP/12345-00000016", "") in new stack
[Mar 14 00:01:29] NOTICE[2790]: res_rtp_asterisk.c:2358 ast_rtp_read: Unknown RTP codec 126 received from '169.254.38.82:20338'
-- Executing [123#from-internal:2] AGI("SIP/12345-00000016", "test.py") in new stack
-- Launched AGI Script /var/lib/asterisk/agi-bin/test.py
test.py: Failed to execute '/var/lib/asterisk/agi-bin/test.py': No such file or directory
-- Auto fallthrough, channel 'SIP/12345-00000016' status is 'UNKNOWN'
-- Executing [h#from-internal:1] Hangup("SIP/12345-00000016", "") in new stack
== Spawn extension (from-internal, h, 1) exited non-zero on 'SIP/12345-00000016'
i see that i have this file in /var/lib/asterisk/agi-bin which 777 permission.
Please suggest to get this basic script work.
thank all in advance
There can be 2 reason for such error:
no file /var/lib/asterisk/agi-bin/test.py in that location or it not accessable by asterisk for some reason - linux general permission issue.
file /var/lib/asterisk/agi-bin/test.py, but no some libraries(python give same error). This one is more likly. Try execute script manual from asterisk user and see what it say.
Recomendation: Use FastAGI instead of AGI.