I am new to Kafka but have seen a few tutorials so I know how Kafka works. I am trying to run a producer that I have written in Python but I don't know how to run this file after I have started my zookeeper server and kafka server. If anyone can tell me the structure of the command that is to be written in command prompt, I would really appreciate it.
Thanks!
Kafka Producer:
import json
import time
from kafka import KafkaProducer
from kafka.errors import KafkaError
from kafka.future import log
if __name__ == "__main__":
producer = KafkaProducer(bootstrap_servers= 'localhost: 9092')
future = producer.send('my-topic', b"test")
try:
record_metadata = future.get( timeout=10)
except KafkaError :
log.exeption()
pass
print( record_metadata.topic)
print(record_metadata.partition)
print(record_metadata.offset)
producer = KafkaProducer(value_serializer = lambda m: json.dumps(m).encode('ascii'))
producer.send('json-topic',{'key':'value'})
for _ in range (100):
producer.send('my-topic', b"test")
producer.send('my-topic',b"\xc2Hola, mundo!")
time.sleep(1)
So your question is how to run a python script? Simply save it, make executable and execute:
chmod +x ./kProducer.py
python ./kproducer.py
More detail are here: How to Run a Python Script via a File or the Shell
Add shebang line at the top to your script:
#!/usr/bin/env python-version
Replace the python-version with python2 for 2.x and python3 for 3.x
To check the version of python use the command:
python -V
The shebang line will determine the script ability to run as standalone. This will help when you want to double-click the script and execute it and not from the terminal. or simply say
python scriptname.py
Related
I am new to the nifi platform.
I am trying to use a python script to capture network packet which works on VScode and want to implement same script using NiFi but unable to do so.
This python code I used:
import os, subprocess
from subprocess import PIPE
from datetime import datetime
n = 10
filename = str(datetime.now()).replace(" ","")
b = subprocess.run(f'sudo tcpdump udp -e -i wlp6s0 -nn -vvv -c {n} -w {filename}.raw',shell=True)
c = '"X%02x"'
a = subprocess.run(f"sudo hexdump -v -e '1/1 {c}' {filename}.raw| sed -e 's/\s\+//g' -e 's/X/\\x/g' ", shell=True , stdout=PIPE, stderr=PIPE)
output_file = open (f'{filename}.txt', 'w')
output_file.write(str(a.stdout))
# print("*************************File Created*************************")
output_file.close()
I am using Execute Script Processor for implementing the python script. But it doesn't seem to be working. For executing the "sudo command" I have set to use no password so that no input is needed while executing the script.
Thank you!
Since you're just calling shell commands, you might consider ExecuteStreamCommand instead. You can still run the top-level Python script to call the subprocesses, but since you're not working with flowfile attributes you might be better served being able to call "real" Python. In ExecuteScript the engine is actually Jython and it doesn't let you import native (CPython) modules such as scikit, you can only import pure Python modules (Python scripts that don't themselves import native modules)
I have a problem with .bat file. There is a .bat file that starts the server. But on a Mac OS system, this is impossible. Are there any options to rewrite it to python or bash so that it would be possible to start from MacBook?
This is .bat file:
echo start web server..
start cmd /k node webServer.js
echo start chrome..
start chrome.exe /k http://localhost:8080
Thanks for helping!
Ok, so it would be better to use bash scripts. They are much more powerful than bat and they work on all Unix like OS-s (Linux, Mac ..) and can work on windows with some modifications.
This will show you how to run node:
Running node from a bash script
This will show you how to run the app:
https://askubuntu.com/questions/682913/how-to-write-shell-script-to-start-some-programs
Also, look at this link for an introduction to bash, it is a good thing to know it:
https://linuxconfig.org/bash-scripting-tutorial-for-beginners
Also on the https://www.mac-forums.com/forums/switcher-hangout/302162-execute-bat-file-mac.html you can see how to run it on mac but as they pointed out there it doesn't work 100%.
Edit1:
This is the code:
#!/bin/bash
echo "Star server .."
node webServer.js
echo "Open chrome"
open http://localhost:8080
For node just add the path to file, like you would usually run it.
For last line it opens default browser with link.
Here's a Python example that is cross-platform (unless you don't have node in PATH) and using only the standard library:
# client.py
import subprocess
import webbrowser
if __name__ == '__main__':
try:
server_proc = subprocess.Popen(['node', 'webServer.js'])
webbrowser.open('http://localhost:8080')
server_proc.communicate()
except KeyboardInterrupt:
server_proc.terminate()
Note, however, that webbrowser.open will open the browser that is set as default, so it could be Safari or whatever. If you want to specifically open Chrome, you will have to pass the full path to the executable (or modify your PATH env var). Example:
# client.py
import os
import subprocess
if __name__ == '__main__':
try:
server_proc = subprocess.Popen(['node', 'webServer.js'])
chrome_exe = os.path.join('/', 'Applications', 'Google Chrome.app', 'Contents', 'MacOS', 'Google Chrome')
subprocess.Popen([chrome_exe, 'http://localhost:8080'])
server_proc.communicate()
except KeyboardInterrupt:
server_proc.terminate()
I want to execute commands in the terminal through a python scripts.
i want to create a script which takes data from a .txt file adds that in a list and then one by one execute them in the terminal.
what i am looking for is a process to execute commands in the terminal in Kali Linux, I couldn't find anything online.
like in windows we use import subprocess or import os
Thank you.
example command is like
python3 app.py
Try this:
import subprocess
command = "python3 app.py"
subprocess.call(command, shell=True)
You can use the os.system function. It returns the return value of the command run.
E.g.,
status = os.system('echo hello')
I have got recommendations to run Python script on Linux Server using this sequence:
1) source path to environment
2) python path to python script
So, I need to call this script from php command system:
Should it be like this?
system("source *path to environment* python *path to python script*")
How to set environment and then run script?
system("*path/to/environment*/bin/python *path to python script*")
is enough. The trick is to run the script using python from the environment; that way you don't need to source the activation script.
Your python script need to contain 'shebang' string at the top #!/usr/bin/env python
for example file start.py:
#!/usr/bin/env python
def some_func():
print("Run some python function")
if __name__ == "__main__":
some_func()
then also you need to make it executable with command:
chmod +x start.py
and you can run it:
./start.py
I am trying to run a python script at ubuntu startup.
The script(play.py) plays a youtube video in chrome browser.
import webbrowser
import time
import subprocess as sp
val = 0;
while(val <1):
time.sleep(1)
#webbrowser.open("https://www.youtube.com/watch?v=dO1rMeYnOmM")
child = sp.Popen("google-chrome %s" % "https://www.youtube.com/watch?v=dO1rMeYnOmM", shell=True)
val = val +1
To run this script at ubuntu startup I added the following entry in rc.local:
python /home/user_name/Desktop/python/play.py &
And changes the mode of play.py to 777.
I also tried copying this script file to /etc/init.d.
But still the script does not execute at system startup.
Try sudo /usr/bin/python /home/user_name/Desktop/python/play.py & instead.