Develop Reference

How do I create an Infinite Loop in Python that checks if a variable from a request and a local variable are different and trigger a block

I am trying to make a Python script that runs as a background service with NSSM that will only preform an action when a new version of a program is released.
To do this I want to check a web URL with requests every x amount of time and only if the version the server reports has changed from the local version run a updater block.
I currently have the block that does the update working fine but I just don't know how I would compare the request result and local version info from a file and only run that block if they don't match.
I get the newest version on the server with this code:
winclientsettings = requests.get('https://myapi.website/client-version/Windows')
winclientdict = winclientsettings.json()
serverversion = list(winclientdict.values())[1]
And the local version with:
appdata = os.getenv('LOCALAPPDATA')
programfolder = appdata + '/My Program'
versionfile = programfolder + '/localversion'
with open(versionfile, "r") as file:
localversion = file.readline()
I want to put these together into an infinite loop, where if serverversion and localversion are the same it just does nothing and waits a set amount of time before repeating, and if serverversion and localversion are different, will trigger a block higher in the file.

Put what you have in a while loop, and use the builtin time.sleep
import os
import time
import requests
WAIT_SECONDS = 300
appdata = os.getenv('LOCALAPPDATA')
programfolder = appdata + '/My Program'
versionfile = programfolder + '/localversion'
def get_server_version():
winclientsettings = requests.get(
'https://myapi.website/client-version/Windows')
winclientdict = winclientsettings.json()
serverversion = list(winclientdict.values())[1]
return serverversion
def get_local_version():
with open(versionfile, "r") as file:
localversion = file.readline()
return localversion
def update_if_new():
server_version = get_server_version()
local_version = get_local_version()
if server_version != local_version:
with open(versionfile, "w") as file:
file.write(server_version)
def main():
while True:
update_if_new()
time.sleep(WAIT_SECONDS)
if __name__ == '__main__':
main()

Pyrebase4 storage No value for argument 'filename'

I have been struggling with installing pyrebase on my second PC, which has the same python version 3.8.2 as my main PC, my main PC has this pyrebase script working properly
from pyrebase import pyrebase
import os
import time
project_root = os.path.dirname(os.path.dirname(__file__))
keys_path = os.path.join(project_root, 'keys')
hylKeyPath = os.path.join(keys_path, 'ServiceAccountKey.json')
firebase = pyrebase.initialize_app({
"apiKey": "aksdjalksjdlkajsdlkjalkdja",
"authDomain": "lkjsakdjlkjsad.firebaseapp.com",
"databaseURL": "https://asdlkasjldkjaslkd.firebaseio.com",
"storageBucket": "asdasdadjaslkjhd.appspot.com",
"serviceAccount": asdKeyPath
})
storage = firebase.storage()
def sleepCountDown(t):
while t > 0:
print(f"sleeping for {t} seconds...")
t -= 1
time.sleep(1)
while True:
print('fetching data')
files = storage.child('/').list_files()
for file in files:
if 'records/' in file.name:
# get the file url path
# print(storage.child(file.name).get_url(None))
# downloads file
storage.child(file.name).download(os.path.basename(file.name))
# deletes file? kinda deletes the entire folder
storage.delete(file.name)
sleepCountDown(10)
But somehow I am not able to install pyrebase on my second PC so I had to install pyrebase4.
But it seems this one has some bugs and keeps highlighting storage.child(file.name).download(os.path.basename(file.name)) saying that storage: No value for argument 'filename' in method call.
then when I run, it says "Crypto has no method or something"
anyone knows what is going on?

Just noticed the syntax was different on pyrebase4, it's .download(path, filename) the documentation was outdated.

Instead of:
storage.child(file.name).download(os.path.basename(file.name))
Use:
storage.child("filename").download(filename="filename" , path="E:/ddf/")
Where E:/ddf/ is path you want to save your file.

python3 running old version of script

I've changed a script by commenting out a section and adding some print statements above it for testing purposes, the problem is when I run the script from any other directory than the one it's in, python is running the old version of the script.
Clearing __pycache__ had no effect.
Here's the python script in question:
import discord
import watson
import configparser
import os
from discord.ext.commands import Bot
from getpass import getuser
print("WHY ISN'T THIS BEING CALLED!!")
#print(os.getcwd())
#os.chdir("/home/{}/discordBotStaging".format(getuser()))
#print(os.getcwd())
#if(not os.path.isfile("./config.ini")):
# print("No config file found, make sure you have one in the same directory as this python script\nexiting")
# quit()
config = configparser.ConfigParser()
config.read("./config.ini")
TOKEN = config["DEFAULT"]["DISCORD_KEY"]
BOT_PREFIX = ("!", "$")
client = Bot(command_prefix=BOT_PREFIX)
#client.command(name="memealyze",
description="When uploading image include command \"!memealyze\" | \"!Memealyze\" | \"!MemeAlyze\" | \"!ma\" as a comment",
brief="Neural network put to good use",
aliases=["Memealyze", "MemeAlyze", "ma"],
pass_context=True)
async def GetMemeContents(context):
await client.say("Sending image to the mothership, hold tight.")
if(not context.message.attachments):
await client.say(
"Couldn't find image attachement. Make sure you include \"!memealyze\" or any of it's variants as a comment when submitting an image")
return
imageUrl = str(context.message.attachments[0]["url"])
messageContent = ""
resultDict = watson.ReturnWatsonResults(imageUrl)
for key,val in resultDict.items():
messageContent += "{} : {}%\n".format(key, val)
await client.say("Done, the boys at IBM said they found this:\n" + messageContent)
client.run(TOKEN)
And here's the issue:
yugnut#RyzenBuild:~$ python3 discordBotStaging/main.py
No config file found, make sure you have one in the same directory as this python script
exiting
yugnut#RyzenBuild:~$
yugnut#RyzenBuild:~/discordBotStaging$ python3 main.py
WHY ISN'T THIS BEING CALLED!!
^Cyugnut#RyzenBuild:~/discordBotStaging$
EDITS:
#ShadowRanger suggestions:
Try moving the print above all the imports in your script.
This yields promising results, I do get output by trying this but right after that I still run into the same issue
You can't use relative paths like that if the config file is expected to be in the same directory as the script; you have to do config.read(os.path.join(os.path.dirname(__file__), 'config.ini'))
I think my ignorance is showing :^), I changed this in my script as well
After making these edits along with trying to run the script after commenting out my import configparser line I still get the same error.

How to fix AttributeError: 'module' object has no attribute 'Client' when running python in Google Cloud Interactive Shell

I'm trying to run a python script that simulates traffic sensors sending in data in real time to PubSub on my Google Cloud Shell. I'm getting this error
Traceback (most recent call last):
File "./send_sensor_data.py", line 87, in <module>
psclient = pubsub.Client()
AttributeError: 'module' object has no attribute 'Client'
Tried running google.cloud.pubsub.__file__, no duplicates exist.
I've been searching everywhere and the popular consensus was to install the pubsub package into a virtual environment which I've tried to no avail.
What I've tried so far:
Set VM to clean state
Uninstalled and reinstalled all gcloud components
Updated all gcloud components to the latest version
uninsalled and reinstalled python pubsub library
Installed pubsub inside a virtualenv
Tried from a different project
Tried from a different GCP account
This is my script:
import time
import gzip
import logging
import argparse
import datetime
from google.cloud import pubsub
TIME_FORMAT = '%Y-%m-%d %H:%M:%S'
TOPIC = 'sandiego'
INPUT = 'sensor_obs2008.csv.gz'
def publish(topic, events):
numobs = len(events)
if numobs > 0:
with topic.batch() as batch:
logging.info('Publishing {} events from {}'.
format(numobs, get_timestamp(events[0])))
for event_data in events:
batch.publish(event_data)
def get_timestamp(line):
# look at first field of row
timestamp = line.split(',')[0]
return datetime.datetime.strptime(timestamp, TIME_FORMAT)
def simulate(topic, ifp, firstObsTime, programStart, speedFactor):
# sleep computation
def compute_sleep_secs(obs_time):
time_elapsed = (datetime.datetime.utcnow() - programStart).seconds
sim_time_elapsed = (obs_time - firstObsTime).seconds / speedFactor
to_sleep_secs = sim_time_elapsed - time_elapsed
return to_sleep_secs
topublish = list()
for line in ifp:
event_data = line # entire line of input CSV is the message
obs_time = get_timestamp(line) # from first column
# how much time should we sleep?
if compute_sleep_secs(obs_time) > 1:
# notify the accumulated topublish
publish(topic, topublish) # notify accumulated messages
topublish = list() # empty out list
# recompute sleep, since notification takes a while
to_sleep_secs = compute_sleep_secs(obs_time)
if to_sleep_secs > 0:
logging.info('Sleeping {} seconds'.format(to_sleep_secs))
time.sleep(to_sleep_secs)
topublish.append(event_data)
# left-over records; notify again
publish(topic, topublish)
def peek_timestamp(ifp):
# peek ahead to next line, get timestamp and go back
pos = ifp.tell()
line = ifp.readline()
ifp.seek(pos)
return get_timestamp(line)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Send sensor data to Cloud Pub/Sub in small groups, simulating real-time behavior')
parser.add_argument('--speedFactor', help='Example: 60 implies 1 hour of data sent to Cloud Pub/Sub in 1 minute', required=True, type=float)
args = parser.parse_args()
# create Pub/Sub notification topic
logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.INFO)
psclient = pubsub.Client()
topic = psclient.topic(TOPIC)
if not topic.exists():
logging.info('Creating pub/sub topic {}'.format(TOPIC))
topic.create()
else:
logging.info('Reusing pub/sub topic {}'.format(TOPIC))
# notify about each line in the input file
programStartTime = datetime.datetime.utcnow()
with gzip.open(INPUT, 'rb') as ifp:
header = ifp.readline() # skip header
firstObsTime = peek_timestamp(ifp)
logging.info('Sending sensor data from {}'.format(firstObsTime))
simulate(topic, ifp, firstObsTime, programStartTime, args.speedFactor)

The pubsub.Client class exists until the 0.27.0 version of the pubsub python package. So I just created a virtual environment and installed the 0.27.0 version of pubsub into it.
Here are the commands:
virtualenv venv
source venv/bin/activate
pip install google-cloud-pubsub==0.27.0

Solution for Google Cloud Platforms is:
Modify the send_senor_data.py file as follows:
a. Comment the original import statement for pub_sub and use _v1 version
#from google.cloud import pubsub
from google.cloud import pubsub_v1
b. Find this code and replace it as follows:
#publisher = pubsub.PublisherClient()
publisher = pubsub_v1.PublisherClient()
Then execute your send_sensor_data.py as follows:
./send_sensor_data.py --speedFactor=60 --project=YOUR-PROJECT-NAME

There's no pubsub.Client class. You
need to choose a PublisherClient or SubscriberClient
see https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/pubsub/google/cloud/pubsub.py

Python Script works when run from command line but not when run from windows service

I have created a windwos service utilising the following code:
import win32service
import win32serviceutil
import win32api
import win32con
import win32event
import win32evtlogutil
import os, sys, string, time
class aservice(win32serviceutil.ServiceFramework):
_svc_name_ = "PAStoDistillerIFC"
_svc_display_name_ = "PAS DW to Distiller Interface"
_svc_description_ = "Service that checks the Clinical Research folder for any new files from PAS to process in Distiller"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
import servicemanager
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,servicemanager.PYS_SERVICE_STARTED,(self._svc_name_, ''))
#self.timeout = 640000 #640 seconds / 10 minutes (value is in milliseconds)
self.timeout = 120000 #120 seconds / 2 minutes
# This is how long the service will wait to run / refresh itself (see script below)
while 1:
# Wait for service stop signal, if timeout, loop again
rc = win32event.WaitForSingleObject(self.hWaitStop, self.timeout)
# Check to see if self.hWaitStop happened
if rc == win32event.WAIT_OBJECT_0:
# Stop signal encountered
servicemanager.LogInfoMsg("PAStoDistillerIFC - STOPPED!") #For Event Log
break
else:
#[actual service code between rests]
try:
file_path = "D:\\SCRIPTS\\script.py"
execfile(file_path) #Execute the script
except:
servicemanager.LogInfoMsg("File CRASHED")
pass
#[actual service code between rests]
def ctrlHandler(ctrlType):
return True
if __name__ == '__main__':
win32api.SetConsoleCtrlHandler(ctrlHandler, True)
win32serviceutil.HandleCommandLine(aservice)
To run this script:
import os, re, urllib, urllib2, time, datetime
def postXML( path, fname):
fileresultop = open("D:\\CLinicalResearch\\SCRIPTS\\LOG.txt", 'a') # open result file
fileresultop.write('CheckXXX ')
fileresultop.close()
now = datetime.datetime.now() #####ALWAYS CRASHES HERE######
fileresult = open("D:\\SCRIPTS\\IFCPYTHONLOG.txt", 'a') # open result file
fileresultop = open("D:\\SCRIPTS\\LOG.txt", 'a')
fileresultop.write('Check2 ')
fileresultop.close()
path="D:\\Test2" # Put location of XML files here.
procpath="D:\\Test2Processed" # Location of processed files
now = datetime.datetime.now()
dirList=os.listdir(path)
for fname in dirList: # For each file in directory
if re.search("PatientIndexInsert", fname): # Brand new patient records
fileresultop = open("D:\\SCRIPTS\\LOG.txt", 'a') # open result file
fileresultop.write('Check1 ')
fileresultop.close()
postXML(path, fname)
I have pared down the script to the bare code where I believe this is crashing.
This works perfectly from the command line, I run the windows service under my own login.
Once I take the datetime function out of the function it seems to work.
Edit 1: I saw that the service runs in a blank environment. I don't have any environmental variables set myself.
Edit 2: Added traceback:
File "D:\ClinicalResearch\SCRIPTS\PAS2DIST.py", line 23, in <module>
postXML(path, fname)
File "D:\ClinicalResearch\SCRIPTS\PAS2DIST.py", line 6, in postXML
now = datetime.datetime.now()
NameError: global name 'datetime' is not defined

I didn't find the cause but I did find a workaround.
I needed to import all the same libraries into the function too. Once I did that, worked like a charm.
Hope this can help someone else.