Develop Reference

How to launch GCP Compute Engine VM with Startup Script via Python API?

GCP has a published create_instance() code snippet available here, which I've seen on SO in a couple places e.g. here. However, as you can see in the first link, it's from 2015 ("Copyright 2015 Google Inc"), and Google has since published another code sample for launching a GCE instance dated 2022. It's available on github here, and this newer create_instance function is what's featured in GCP's python API documentation here.
However, I can't figure out how to pass a startup script via metadata to run on VM startup using the modern python function. I tried adding
instance_client.metadata.items = {'key': 'startup-script',
'value': job_script}
to the create.py function (again, available here along with supporting utility functions it calls) but it threw an error that the instance_client doesn't have that attribute.
GCP's documentation page for starting a GCE VM with a startup script is here, where unlike most other similar pages, it contains code snippets only for console, gcloud and (REST)API; not SDK code snippets for e.g. Python and Ruby that might show how to modify the python create_instance function above.
Is the best practice for launching a GCE VM with a startup script from a python process really to send a post request or just wrap the gcloud command
gcloud compute instances create VM_NAME \
--image-project=debian-cloud \
--image-family=debian-10 \
--metadata-from-file=startup-script=FILE_PATH
...in a subprocess.run()? To be honest I wouldn't mind doing things that way since the code is so compact (the gcloud command at least, not the POST request way), but since GCP provides a create_instance python function I had assumed using/modifying-as-necessary that would be the best practice from within python...
Thanks!

So, the simplest (!) way with the Python library to create the equivalent of --metadata-from-file=startup-scripts=${FILE_PATH} is probably:
from google.cloud import compute_v1
instance = compute_v1.Instance()
metadata = compute_v1.Metadata()
metadata.items = [
{
"key":"startup-script",
"value":'#!/usr/bin/env bash\necho "Hello Freddie"'
}
]
instance.metadata = metadata
And another way is:
metadata = compute_v1.Metadata()
items = compute_v1.types.Items()
items.key = "startup-script"
items.value = """
#!/usr/bin/env bash
echo "Hello Freddie"
"""
metadata.items = [items]
NOTE In the examples, I'm embedding the content of the FILE_PATH in the script for convenience but you could, of course, use Python's open to achieve a more comparable result.
It is generally always better to use a library|SDK if you have one to invoke functionality rather than use subprocess to invoke the binary. As mentioned in the comments, the primary reason is that language-specific calls give you typing (more in typed languages), controlled execution (e.g. try) and error handling. When you invoke a subprocess its string-based streams all the way down.
I agree that the Python library for Compute Engine using classes feels cumbersome but, when you're writing a script, the focus could be on the long-term benefits of more explicit definitions vs. the short-term pain of the expressiveness. If you just wanna insert a VM, by all means using gcloud compute instances create (I do this all the time in Bash) but, if you want to use a more elegant language like Python, then I encourage you to use Python entirely.
CURIOSITY gcloud is written in Python. If you use Python subprocess to invoke gcloud commands, you're using Python to invoke a shell that runs Python to make a REST call ;-)

when using Watchman's watch-make I want to access the name of the changed files

I am writing a watchman command with watchman-make and I'm at a loss when trying to access exactly what was changed in the directory. I want to run my upload.py script and inside the script I would like to access filenames of newly created files in /var/spool/cups-pdf/ANONYMOUS .
so far I have
$ watchman-make -p '/var/spool/cups-pdf/ANONYMOUS' -—run 'python /home/pi/upload.py'
I'd like to add another argument to python upload.py so I can have an exact filepath to the newly created file so that I can send the new file over to my database in upload.py,
I've been looking at the docs of watchman and the closest thing I can think to use is a trigger object. Please help!

Solution with watchman-wait:
Assuming project layout like this:
/posts/_SUBDIR_WITH_POST_NAME_/index.md
/Scripts/convert.sh
And the shell script like this:
#!/bin/bash
# File: convert.sh
SrcDirPath=$(cd "$(dirname "$0")/../"; pwd)
cd "$SrcDirPath"
echo "Converting: $SrcDirPath/$1"
Then we can launch watchman-wait like this:
watchman-wait . --max-events 0 -p 'posts/**/*.md' | while read line; do ./Scripts/convert.sh $line; done
When we changing file /posts/_SUBDIR_WITH_POST_NAME_/index.md the output will be like this:
...
Converting: /Users/.../Angular/dartweb_quickstart/posts/swift-on-android-building-toolchain/index.md
Converting: /Users/.../Angular/dartweb_quickstart/posts/swift-on-android-building-toolchain/index.md
...

watchman-make is intended to be used together with tools that will perform a follow-up query of their own to discover what they want to do as a next step. For example, running the make tool will cause make to stat the various deps to bring things up to date.
That means that your upload.py script needs to know how to do this for itself if you want to use it with watchman.
You have a couple of options, depending on how sophisticated you want things to be:
Use pywatchman to issue an ad-hoc query
If you want to be able to run upload.py whenever you want and have it figure out the right thing (just like make would do) then you can have it ask watchman directly. You can have upload.py use pywatchman (the python watchman client) to do this. pywatchman will get installed if the the watchman configure script thinks you have a working python installation. You can also pip install pywatchman. Once you have it available and in your PYTHONPATH:
import pywatchman
client = pywatchman.client()
client.query('watch-project', os.getcwd())
result = client.query('query', os.getcwd(), {
"since": "n:pi_upload",
"fields": ["name"]})
print(result["files"])
This snippet uses the since generator with a named cursor to discover the list of files that changed since the last query was issued using that same named cursor. Watchman will remember the associated clock value for you, so you don't need to complicate your script with state tracking. We're using the name pi_upload for the cursor; the name needs to be unique among the watchman clients that might use named cursors, so naming it after your tool is a good idea to avoid potential conflict.
This is probably the most direct way to extract the information you need without requiring that you make more invasive changes to your upload script.
Use pywatchman to initiate a long running subscription
This approach will transform your upload.py script so that it knows how to directly subscribe to watchman, so instead of using watchman-make you'd just directly run upload.py and it would keep running and performing the uploads. This is a bit more invasive and is a bit too much code to try and paste in here. If you're interested in this approach then I'd suggest that you take the code behind watchman-wait as a starting point. You can find it here:
https://github.com/facebook/watchman/blob/master/python/bin/watchman-wait
The key piece of this that you might want to modify is this line:
https://github.com/facebook/watchman/blob/master/python/bin/watchman-wait#L169
which is where it receives the list of files.
Why not triggers?
You could use triggers for this, but we're steering folks away from triggers because they are hard to manage. A trigger will run in the background and have its output go to the watchman log file. It can be difficult to tell if it is running, or to stop it running.
The interface is closer to the unix model and allows you to feed a list of files on stdin.
Speaking of unix, what about watchman-wait?
We also have a command that emits the list of changed files as they change. You could potentially stream the output from watchman-wait in your upload.py. This would make it have some similarities with the subscription approach but do so without directly using the pywatchman client.

How do I get feedback from T32 after running cmm scripts?

I am running a T32 CMM script as below via command line(putting in a python wrapper) however I would like to know the status of T32 whether the script has been running successfully or was there an error,how can I that feedback from T32?
cd C:\T32\bin\windows64
Config.t32:
RCL=NETASSIST
PORT=20000
PACKLEN=1024
; Environment variables
OS=
ID=T32
TMP=C:\Users\jhigh\AppData\Local\Temp
SYS=C:\T32
PBI=
USB
; Printer settings
PRINTER=WINDOWS
USAGE:-
t32marm.exe -s c:\Temp\vi_chip_cmd_line.cmm \\Filerlocation\data\files

The TRACE32 "API for Remote Control and JTAG Access" allows you to communicate with a running TRACE32 application.
To enable the API for your TRACE32 application, just add the following two lines to your TRACE32 start-configuration file ("config.t32"). Empty lines before and after the two lines are mandatory.
RCL=NETASSIST
PORT=20000
The usage of the API is described in the PDF api_remote.pdf, which is in the PDF folder of your TRACE32 installation or you can download it from http://www.lauterbach.com/manual.html
You can find examples on how to use the remote API with Python at http://www.lauterbach.com/scripts.html (Just search the page for "Python")
To check if your PRACTICE script ("vi_chip_cmd_line.cmm") is still running, use the API function T32_GetPracticeState();
I also suggest to create an artificial variable in the beginning of your script with Var.NEWGLOBAL int \state. During your scripted test, set the variable "\state" to increasing values with Var.Set \state=42. Via the TRACE32 command EVAL Var.VALUE(\state) and API call T32_EvalGet() you can get the current value of the variable "\state" and by doing so, you can check, if your script reached its final state.
Another approach would be to write a log-file from your PRACTICE script ("vi_chip_cmd_line.cmm") by using the TRACE32 command APPEND and read the log file from your Python script.

Please check your T32 installation for a demo on how to use the T32 API (demo/api/python). Keep in mind that it will not work without a valid license. It's also important that if you use Python inside 32-bit cygwin on a 64-bit host you need to load the 32-bit DLL.
Configuration:
RCL=NETASSIST
PORT=20000
PACKLEN=1024
Python script:
import platform
import ctypes
# Adjust the path / name to the DLL
t32api = ctypes.CDLL("./t32api64.dll")
t32api.T32_Config(b"NODE=",b"localhost")
t32api.T32_Config(b"PORT=",b"20000")
t32api.T32_Config(b"PACKLEN=",b"1024")
t32api.T32_Init()
t32api.T32_Attach(1)
t32api.T32_Ping()
t32api.T32_Cmd(b"AREA")
t32api.T32_Exit()
Then you can use the commands / techniques that Holger has suggested:
T32_GetPracticeState()
to get the current run-state of PRACTICE. And / or set a variable inside your script
Var.Assign \state=1
Var.Assign \state=2
....
and then poll it using T32_ReadVariableValue()

Communicating between Autohotkey and python

Is there a way to send some parameter from autohotkey to python.
Using Autohot key I read some number from the notepad and store in a variable and now I want to send this number to the python code in order to do some calculations.
My Autohotkey code is:
controlGetText, telphoneNumber, Edit1, Untitled - Notepad
And I want to send this telphoneNumber to python file.
Is there a way I can do that?
Do I need to create an exe file of a python and then call from autohotkey?
For example:
RunWait, C:\Button\button.exe telphoneNumber
Or do I need to run command prompt commands from autohotkey to run python program? Something like:
Run Cmd Python C:\Button\button.py telphoneNumber
I do not know which is the best way as I am newbie in Autohotkey.
Any suggestion will be appreciated.
EDIT:
However I succeded in sending parameter by using run command from autohotkey, which will execute the python file from command prompt.
Run Cmd \k "Python C:\Button\button.py %telphoneNumber%"
But still want to know if this is the right solution, or if there are others?

Inter-process communication would be capable of sending the information while the Python script is already running.
Forum thread: http://www.autohotkey.com/forum/topic21699.html (there's a nice documentation link in that post)
You could also use TCP/IP Network communication (like in the post below), but that probably wouldn't be quite as slick as using IPC.
Forum thread: http://www.autohotkey.com/forum/topic13829.html

The way you got it working is the easiest, and probably best, method of accomplishing what you want.
Communication between applications can be done with more methods then you probably can imagine, but as long as it doesn't have to be realtime you can call your programs with arguments, as it is easy and reliable.

Python COM server allows directly calling Python functions(with args and return) using AHK.
you use it like this: MsgBox % pythonComServer.method(args)
You do not need to have a python script already running.
ComObjCreate() will instantiate an instance of python.
I don't know how the inter-process communication is done in the background by pywin32, but using it is simple.
2 examples here: Call python function with arguments and get returned value in autohotkey

Using other languages with ruby

Is it possible to use multiple languages along side with ruby. For example, I have my application code in Ruby on Rails. I would like to calculate the recommendations and I would like to use python for that. So essentially, python code would get the data and calculate all the stuff and probably get the data from DB, calculate and update the tables.Is it possible and what do you guys think about its adv/disadv
Thanks

If you are offloading work to an exterior process, you may want to make this a webservice (ajax, perhaps) of some sort so that you have some sort of consistent interface.
Otherwise, you could always execute the python script in a subshell through ruby, using stdin/stdout/argv, but this can get ugly quick.

Depending on your exact needs, you can either call out to an external process (using popen, system, etc) or you can setup another mini-web-server or something along those lines and have the rails server communicate with it over HTTP with a REST-style API (or whatever best suits your needs).
In your example, you have a ruby frontend website and then a number-crunching python backend service that builds up recommendation data for the ruby site. A fairly nice solution is to have the ruby site send a HTTP request to the python service when it needs data updating (with a payload of information to identify what it needs doing to what or some such) and then the python backend service can crunch away and update the table which presumably your ruby frontend will automatically pick up the changes of during the next request and display.

I would use the system command
as such
system("python myscript.py")

An easy, quick 'n' dirty solution in case you have python scripts and you want to execute them from inside rails, is this:
%x[shell commands or python path/of/pythonscript.py #{ruby variables to pass on the script}]
or
``shell commands or python path/of/pythonscript.py #{ruby variables to pass on the script}\ (with ` symbol in the beginning and the end).
Put the above inside a controller and it will execute.
For some reason, inside ruby on rails, system and exec commands didn't work for me (exec crashed my application and system doesn't do anything).