Develop Reference

Looking for general feedback to improve (python) programming skills: script generates resource pulling list for an Ark Survival Evolved mod (S+)

I am somewhat new to programming. I did not go to school to learn about it, but do read a lot online to learn more about coding.
In general, if I know a concept then I can usually figure it out with google.
Obvious from the post, I like to play games and so I thought one day while configuring some settings for a mod, I realized that the information was extremely difficult to dig up. I thought that I could try to solve it by writing a python script (no real success writing scripts before this point).
The following is the general workflow that I wrote up before tackling it in python:
Replace base path up to 'Mods' with '/Game/Mods'
Find all paths to files in Mods folder
Rename all mod files ending in uasset. E.g. filname.uasset -> filename.filename
Parse Mod's name from binary file in Mod subdirectory, replace integer name with text name for all related paths. E.g. mod name 899987403 -> Primal_Fear_Bosses
Example before (ubuntu path):
/mnt/c/Program Files (x86)/Steam/steamapps/common/ARK/ShooterGame/Content/Mods/899987403/Vanilla/Projectiles/Manticore/Proj/PFBProjManticoreQuill.uasset
& after with script: /Game/Mods/Primal_Fear_Bosses/Vanilla/Projectiles/Manticore/Proj/PFBProjManticoreQuill.PFBProjManticoreQuill,
*Note: align the "/Mods/" section for both lines to see the key difference.
Anyways, I was familiar with a lot of concepts at that point, so I did a lot of googling and wrote the following script (first ever), "Generate_Resource_Pulling_List_for_S+.py":
import os
import sys
from fnmatch import fnmatch
import re
# define the directory that mods are installed in
# ubuntu path
# modDir = "C:/Program Files (x86)/Steam/steamapps/common/ARK/ShooterGame/Content/Mods"
# windows path
modDir = "C:\Program Files (x86)\Steam\steamapps\common\ARK\ShooterGame\Content\Mods\\"
stringPattern = r"\w+(.mod)"
regexPattern = re.compile(stringPattern)
modFolders = [mod for mod in os.listdir(modDir) if not regexPattern.match(mod)]
# loop through mod list and append directory to mod installation path
for items in modFolders:
modPath = "%s%s" % (modDir, items)
# parse mod name from meta file, store for later
modMetaFilePath = "%s\%s" % (modPath, "modmeta.info")
validPath = os.path.exists(modMetaFilePath)
if not validPath:
print('x is:', validPath)
pass
try:
modFile = open(modMetaFilePath, "rb")
binaryContent = modFile.read(-1)
modFile.close()
# if len == 0:
# print('length:', len(binaryContent))
# break
# print(type(binaryContent))
text = binaryContent.decode('UTF-8')
try:
parsedModName = text.split('Mods/')[1].split('/')[0]
if not parsedModName:
break
except ValueError as e:
pass
except Exception as e:
pass
for path, subdirs, files in os.walk(modPath):
# for i in range(len(subdirs)):
# print('Number of Sub Directories: ', len(subdirs))
# print('Current Directory Number in ', path, ': ', subdirs[i])
for name in files:
pattern = "*.uasset"
if fnmatch(name, pattern):
try:
path = os.path.normpath(path) + '\\'
if not path:
continue
try:
basePath = os.path.join('\\Game\\Mods\\', parsedModName)
except Exception as e:
pass
print('failed here:', str(e))
strippedBasePath = os.path.dirname(path).split(items)[1]
if not strippedBasePath:
print('failed at this point stripped path', strippedBasePath, '\n\t', 'path', path)
continue
revisedFileName = os.path.join(
os.path.basename(name).split('.')[0] + '.' + os.path.basename(name).split('.')[0] + ',')
finalPath = "%s%s\%s" % (basePath, strippedBasePath, revisedFileName)
flippedSlashFinalPath = finalPath.replace("\\", "/")
print(flippedSlashFinalPath)
with open("out.txt", "a") as external_file:
add_text = flippedSlashFinalPath
external_file.write(add_text)
external_file.close()
except Exception as e:
print('Something happened', str(e))
I initially installed an Ubuntu environment on Windows because I am unfamiliar with command line/bash/scripting in Windows as seen in the mod path (slashes are reversed and commands are different). I am much more familiar with MacOS and generally work in Linux environments and terminals.
I thought that this script would be usable by others with a bit of programming knowledge, but it is not the most user friendly.
In any case, this was my first attempt at writing something and trying to use good practices.
As another side project to practice more coding, I rewrote it and implemented more complex concepts.
Here is the refactored version:
import os
import sys
from fnmatch import fnmatch
import re
import winreg
stringPattern = r"\w+(.mod)"
regexPattern = re.compile(stringPattern)
class Mod(object):
def __init__(self):
pass
self.steam_path = None
self.mod_folders = []
self.mod_path = None
self.mod_path_list = []
self.mod_meta_path_list = []
self.full_mod_path = None
self.mod_meta = None
self.resource_path = None
#property
def GetSteamPath(self):
try:
steam_key = "SOFTWARE\WOW6432Node\Valve\Steam"
hkey = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, steam_key)
except ValueError as e:
hkey = None
print(e.sys.exc_info())
try:
steam_path = winreg.QueryValueEx(hkey, "InstallPath")
except ValueError as e:
steam_path = None
print(e.sys.exc_info())
winreg.CloseKey(hkey)
self.steam_path = steam_path[0]
return self.steam_path
def GetArkModPath(self):
mod_dir = self.GetSteamPath[:] + "\steamapps\common\ARK\ShooterGame\Content\Mods\\"
self.mod_folders = [mod for mod in os.listdir(mod_dir) if not regexPattern.match(mod)]
mod_path_list = []
count = 1
while count < len(self.mod_folders):
for mod in self.mod_folders:
self.full_mod_path = ("%s%s\\" % (mod_dir, mod))
mod_path_list.append(self.full_mod_path)
self.mod_path_list = mod_path_list
if not mod:
break
# else:
# print('Test1', format(mod_path_list))
return self.mod_path_list
count += 1
def GetModResourceList(self, mod_path_list):
mod_folder_index = 0
for mod_path in mod_path_list:
mod_meta_path = "%s%s" % (mod_path, "modmeta.info")
self.mod_meta_path_list.append(mod_meta_path)
validPath = os.path.exists(mod_meta_path)
if not validPath:
# print('No Mod Meta File found at: ', mod_meta_path)
continue
try:
mod_file = open(mod_meta_path, "rb")
binary_content = mod_file.read(-1)
mod_file.close()
text = binary_content.decode('UTF-8')
try:
parsed_mod_name = text.split('Mods/')[1].split('/')[0]
if not parsed_mod_name:
break
except ValueError as e:
pass
except Exception as e:
pass
for path, subdirs, files in os.walk(mod_path):
# for i in range(len(subdirs)):
# print('Number of Sub Directories: ', len(subdirs))
# print('Current Directory Number in ', path, ': ', subdirs[i])
for uasset_file in files:
pattern = "*.uasset"
if fnmatch(uasset_file, pattern):
try:
path = os.path.normpath(path) + '\\'
if not path:
continue
try:
base_path = os.path.join('\\Game\\Mods\\', parsed_mod_name)
except Exception as e:
pass
print('failed here:', str(e))
stripped_base_path = os.path.dirname(path).split(self.mod_folders[mod_folder_index])[1]
resource_name = os.path.join(
os.path.basename(uasset_file).split('.')[0] + '.' + os.path.basename(uasset_file).split('.')[0] + ',')
full_path = "%s%s\%s" % (base_path, stripped_base_path, resource_name)
resource_path = full_path.replace("\\", "/")
self.resource_path = resource_path
# to see what text is written to the file, uncomment print here
print(self.resource_path)
with open("test_out.txt", "a") as external_file:
add_text = self.resource_path
external_file.write(add_text)
external_file.close()
except Exception as e:
print('Error: ', str(e))
# return self.resource_path[]
mod_folder_index += 1
ark = Mod()
ark_mods = ark.GetArkModPath()
ark.GetModResourceList(ark_mods)
This revised one is much more user friendly and does not necessarily require input or modification of variables. I tried to avoid requiring arguments passed in a method call because I wanted to automate the script as necessary without any input from user.
If you have Ark installed with some mods, then you may be able to actually use or test the scripts. I'm curious if they work for others. FYI, not all paths are usable. It might be best to delete/remove some or just pick out the ones you want. Can't really figure out the best way to identify this.
I struggled with creating an instance of my class, what attributes to set, what properties to set, setting(?) values for attributes for my class, calling those stored values, and just about everything else you can imagine.
Some things are obvious to me, such as it is pointless to create classes and methods in a script that is not necessarily being re-used. It was still fun and the point was to learn and practice.
In any case, I am open to any feedback because I do not know what is common sense for everything or best practices that others have seen. As a side note, I do versioning with local repos. I have worked with git and bitbucket before.
Preferred topics if not sure:
defensive programming
proper exception handling
SOLID (I'm def lacking here)
Classes (I barely got mine working if its not apparrent)
Methods (seems straight forward to me - if code is going to be copied, put it in a method)
Preferred nomenclature for everything, classes, methods, variables (I googled it, but still don't like how it looks especially when they get long)
Readability (I understand that if code requires comments, its not good)
Cleanliness (I guess this is an art, but I see spaghetti when I look at what I wrote)
Links to resources are welcome! I really appreciate any feedback and look forward to all comments.

How to find steam executable path in Python

What are the best ways to find the Steam install path (for example using registry, possible paths, and the Steam start-menu shortcut?)

This code works for a personal project on Windows.
import os
import winreg
import win32api
def read_reg(ep, p = r"", k = ''):
try:
key = winreg.OpenKeyEx(ep, p)
value = winreg.QueryValueEx(key,k)
if key:
winreg.CloseKey(key)
return value[0]
except Exception as e:
return None
return None
Path1 = "{}\\Microsoft\\Windows\\Start Menu\\Programs\\Steam\\Steam.lnk".format(os.getenv('APPDATA'))
if os.path.exists(Path1):
import sys
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shortcut = shell.CreateShortCut(Path1)
Path1Res = shortcut.Targetpath
else:
Path1Res = False
Path2 = str(read_reg(ep = winreg.HKEY_LOCAL_MACHINE, p = r"SOFTWARE\Wow6432Node\Valve\Steam", k = 'InstallPath'))+r"\steam.exe"
Path3 = str(read_reg(ep = winreg.HKEY_LOCAL_MACHINE, p = r"SOFTWARE\Valve\Steam", k = 'InstallPath'))+r"\steam.exe"
if not os.path.exists(Path2):
Path2 = None
if not os.path.exists(Path3):
Path3 = None
PossiblePaths = [r"X:\Steam\steam.exe", r"X:\Program Files\Steam\steam.exe", r"X:\Program Files (x86)\Steam\steam.exe"]
ValidHardPaths = []
for Drive in win32api.GetLogicalDriveStrings().split('\000')[:-1]:
Drive = Drive.replace(':\\', '')
for path in PossiblePaths:
path = path.replace("X", Drive)
if os.path.exists(path):
ValidHardPaths.append(path)
if len(ValidHardPaths) == 0:
ValidHardPaths = ["None"]
print("Registry64: " + str(Path2)+"|"+ "Registry32: "+ str(Path3)+"|"+ "Start Menu Shortcut: "+ str(Path1Res)+"|"+ "Possible Locations: " + ', '.join(ValidHardPaths)+"|")
As I said earlier, this code was for a personal project, but ill still try to explain the code the best I can.
Method 1: (start menu shortcut) works by first trying to find the steam start menu shortcut, if it exists it will read the destination and add 'steam.exe' to it, then it will check if the path is valid (source: https://stackoverflow.com/a/571573/14132974).
Method 2: (registry) works by attempting to find the steam registry path and reading the key: "InstallPath", adding 'steam.exe' to it, and then checking if the path is valid. It will also do the same using the Steam32 registry path (source: https://tutorialexample.com/python-read-and-write-windows-registry-a-step-guide-python-tutorial/, https://github.com/NPBruce/valkyrie/issues/1056).
Method 3: (possible paths) is fairly simple, there is a list of paths where there is a big chance Steam might be installed, it will check this path for every drive in the system and check if the path is valid (source: https://stackoverflow.com/a/827397/14132974).
Lastly, support:
This code obviously supports having a valid path found with a method, if not it will become 'None', it also supports multiple paths being found in the 'possible paths' method

Python os.system(python "program") not working

I'm writing a program with multiple pieces of code, and don't want someone to have to manually copy-paste the entire directory from an install disk, so I've written something that will do it for them (like an autorun).
However, when I try to launch the program after copying it over to the user's computer, I get:
Process finished with exit code 0
instead of the installer booting the program.
This is the def I'm using:
import os
get_project_base_directory()
check_version()
print jason_dir
new_dir = str(jason_dir+"/Jason_Core/Boot.py")
command = "python " + new_dir
print command
os.system(command)
Why might os.system might not be working? I've also tried popopen and subprocess, but the had the same output (exit code 0).
EDIT: I am running this in a script, not the python command line.
EDIT 2: As requested, Boot.py:
# 3rd party libraries
import os
import subprocess
import shutil
from imp import find_module
# My code to import
basedir = ""
# Get the project base directory (drive located on)
def get_project_base_directory():
global basedir
cwd = os.getcwd()
basedir = "/".join(cwd.split("\\")[:len(cwd.split("\\"))-1])
#print basedir
# Copies libraries from disk drive
def copy_libs():
new_dir = "c:/Downloads/Python_Libraries"
if os.path.exists(new_dir):
shutil.rmtree(new_dir)
old_dir = basedir + "/Libraries/Installed"
#print old_dir
try:
shutil.copytree(old_dir, new_dir)
except:
print " Error copying directories! Try manual copy."
print " Go to:", basedir + "/Libraries/Installed"
# Checks to see if the library is installed
def check_lib_install():
temp_var = True
libs_list = ("fbchat", "gtts", "pyaudio", "googlevoice", "espeak", "pywapi", "speech_recognition", "win10toast") # names of libs
for lib in libs_list:
print "Checking lib installation:", lib
try:
__import__(lib)
print " Lib installed"
print ""
except:
print " Lib not installed"
install_lib(lib, libs_list.index(lib))
print "All libraires installed!"
# Install libraries if they don't exist on this machine.
def install_lib(lib, index):
print " Installing lib:", lib
libs_file_list = ("/fbchat-1.0.19/", "/gTTS-1.2.2/", "/PyAudio-0.2.11/", "/pygooglevoice-master/", "/python-espeak-0.5/", "/pywapi-0.3.8/", "/speech_recognition-master/", "/Windows-10-Toast-Notifications-master/") # path to file
print "Installing:", lib
new_dir = "c:/Downloads/Python_Libraries" + libs_file_list[index]
os.chdir(new_dir)
try:
temp_command = "python setup.py install"
subprocess.call(temp_command)
except:
print lib, "failed installation. Try manual install."
print ""
# Gets version number of local copy of Jason, and then the version number of the one on the disk drive. If the local version is lower, the python files are overwritten. If it doesn't yet exist locally, Jason is installed.
def check_version():
local_version = "0"
jason_dir = "C:/Jason_AI" # Where is jason installed to?
if os.path.exists(jason_dir):
with open(jason_dir+"\System Files\Version Number.txt") as local_jason: # Where is jason installed to?
counter = 0
for line in local_jason:
counter += 1
if counter == 2:
local_version = line
#print local_version
elif counter == 3:
break
install_version_file = basedir + "\System Files\Version Number.txt"
install_version_file = "/".join(install_version_file.split("\\"))
with open(install_version_file) as drive_jason:
counter = 0
for line in drive_jason:
counter += 1
if counter == 2:
install_version = line
#print install_version
if install_version == local_version:
print "Version up to date!"
print "Using version number:", local_version
else:
shutil.rmtree(jason_dir)
print "Version outdated"
print "Using version number:", local_version
elif counter == 3:
break
else:
print "Version outdated"
print "Using version number:", local_version
# Controls the boot process, when done, launches into Listen.get_audio()
def control():
get_project_base_directory()
copy_libs()
check_lib_install()
import System_Logins # Importing this here since the required libraries might not be installed if its the first time running Jason on this machine
import Actions # ^^
import System_Commands # ^^
import Listen # ^^
check_version()
logins = System_Logins.read_logins(basedir)
System_Commands.clear_screen()
while True:
speech = Listen.get_audio()
Actions.sort(speech, logins)
def check_packages():
packages = ["System_Logins", "Actions", "System_Commands", "Listen"]
for package in packages:
try:
__import__(package) #Need to work on
print package + ":", "successfully loaded."
packages[packages.index(package)] = "True"
except ImportError:
print package + ":", "error."
return packages
#Remote login (ie, from install)
def remote_control():
packages = check_packages()
#print packages
if any(word in ("System_Logins", "Actions", "System_Commands", "Listen") for word in packages):
control()
import System_Logins # Importing this here since the required libraries might not be installed if its the first time running Jason on this machine
import Actions # ^^
import System_Commands # ^^
import Listen # ^^
logins = System_Logins.read_logins(basedir)
System_Commands.clear_screen()
while True:
speech = Listen.get_audio()
Actions.sort(speech, logins)
remote_control()

Discover OTA "ports" like Arduino IDE

I'm doing this to to list the available com ports in windows and unix.
Windows:
def listWindowsPorts():
serial_ports = []
has_ports = False
path = 'HARDWARE\\DEVICEMAP\\SERIALCOMM'
try:
reg = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, path,)
has_ports = True
except WindowsError:
pass
if has_ports:
for i in range(128):
try:
name, value, type = winreg.EnumValue(reg, i)
except WindowsError:
pass
else:
serial_ports.append(value)
return serial_ports
An this for UNIX (Linux-OSX):
def listUnixPorts(system):
serial_ports = []
dev_path = '/dev/'
if('osx' in system):
dev_names = ['tty.*', 'cu.*']
else:
dev_names = ['ttyACM*', 'ttyUSB*']
for dev_name in dev_names:
pattern = dev_path + dev_name
serial_ports += glob.glob(pattern)
return serial_ports
The arduino's IDE, besides displaying the COM ports, it has the ability to show the OTA "ports" automatically after configured. I'll like to know if there is a way to do this in python, so I can show serial ports and OTA ports together.
Can someone give an example or a site with this information?
I've looked in the documentation of the pyserial library but until now I haven't found the "autodiscover" feature

The answer to this question is to use a mDNS browser, in my case as I'm using python, I resolved with zeroconf

how to access SHOpenFolderAndSelectItems() by ctypes

I want to build a cross-platform application ,I used a windows API called SHOpenFolderAndSelectItems(). Although I found example called it by pywin32,but pywin32 is not available on Linux , I don't want to call a Windows API on linux,just don't want to make another code version for Linux,so I wonder how to access it by ctypes? yes,this API cannot be called on Linux ,I just want to make it silent in the code so that I can freeze the Python scripts into executables by cx_Freeze without pywin32 module-missing error happend .
from win32com.shell import shell, shellcon
import os
def launch_file_explorer(path, files):
'''
Given a absolute base path and names of its children (no path), open
up one File Explorer window with all the child files selected
'''
folder_pidl = shell.SHILCreateFromPath(path,0)[0]
desktop = shell.SHGetDesktopFolder()
shell_folder = desktop.BindToObject(folder_pidl, None,shell.IID_IShellFolder)
name_to_item_mapping = dict([(desktop.GetDisplayNameOf(item, shellcon.SHGDN_FORPARSING|shellcon.SHGDN_INFOLDER), item) for item in shell_folder])
print(name_to_item_mapping)
to_show = []
for file in files:
if file in name_to_item_mapping:
to_show.append(name_to_item_mapping[file])
# else:
# raise Exception('File: "%s" not found in "%s"' % (file, path))
shell.SHOpenFolderAndSelectItems(folder_pidl, to_show, 0)
p=r'E:\aa'
print(os.listdir(p))
launch_file_explorer(p, os.listdir(p))