This question already has answers here:
Run python script on Database event
(3 answers)
Closed 1 year ago.
I have created a python Code that is supposed to run on some data(to generate data points) as soon as the data is filled into the SQL database.
Is it possible to have a MySQL send a trigger event when a data is added into a table that can be detected by a python script so that action A be taken by the python script?
Appreciate any help on this. Thanks in advance!
Is it possible to have a MySQL send a trigger event when a data is added into a table that can be detected by a python script so that action A be taken by the python script?
No, not directly. Triggers call Stored Procedures, etc. which are written in SQL in that DB engine's specific syntax. They don't call external programs. I'm not aware of any Databases which do.
Wrt not-external programs, MySQL let's you create UDF's in C/C++ which can be used in triggers.
The only way to use your Python code as if it was a trigger is in the same code which executes the insert/update, etc. For example something like:
def insert_into_table(x, y, z):
cursor.execute('INSERT INTO ...')
exec_insert_trigger(cursor)
def exec_insert_trigger(cursor_obj):
"""your trigger code here"""
cursor.execute(...)
Or you can try to convert your Python code to C using Cython and then use that in a MySQL trigger.
Related
I have a question and hope someone can direct me in the right direction; Basically every week I have to run a query (SSMS) to get a table containing some information (date, clientnumber, clientID, orderid etc) and then I copy all the information and that table and past it in a folder as a CSV file. it takes me about 15 min to do all this but I am just thinking can I automate this, if yes how can I do that and also can I schedule it so it can run by itself every week. I believe we live in a technological era and this should be done without human input; so I hope I can find someone here willing to show me how to do it using Python.
Many thanks for considering my request.
This should be pretty simple to automate:
Use some database adapter which can work with your database, for MSSQL the one delivered by pyodbc will be fine,
Within the script, connect to the database, perform the query, parse an output,
Save parsed output to a .csv file (you can use csv Python module),
Run the script as the periodic task using cron/schtask if you work on Linux/Windows respectively.
Please note that your question is too broad, and shows no research effort.
You will find that Python can do the tasks you desire.
There are many different ways to interact with SQL servers, depending on your implementation. I suggest you learn Python+SQL using the built-in sqlite3 library. You will want to save your query as a string, and pass it into an SQL connection manager of your choice; this depends on your server setup, there are many different SQL packages for Python.
You can use pandas for parsing the data, and saving it to a ~.csv file (literally called to_csv).
Python does have many libraries for scheduling tasks, but I suggest you hold off for a while. Develop your code in a way that it can be run manually, which will still be much faster/easier than without Python. Once you know your code works, you can easily implement a scheduler. The downside is that your program will always need to be running, and you will need to keep checking to see if it is running. Personally, I would keep it restricted to manually running the script; you could compile to an ~.exe and bind to a hotkey if you need the accessibility.
I am trying to run a Python UDF directly on Druid. Running the Python function directly on the machines has many advantages, not the least of which avoiding huge data transfers from and to the remote database server.
For simplicity sake, let I have a simple Python function that I would like to run directly inside the Druid system. Here's a sample function:
# Calculates the Inverse of a Matrix
def matrix_inverse(A):
return numpy.linalg.inv(A)
I would like to run this function remotely and directly in Druid (and not on the client's side). The data used in the parameters (A) would be obtained from the database.
How could that be done?
No. Python UDFs are not available...yet.
There are JavaScript user defined functions:
https://druid.apache.org/docs/latest/development/javascript.html
Also consider creating a new feature request at: https://github.com/apache/druid/issues
and/or comment on this one: https://github.com/apache/druid/issues/10180
This question already has answers here:
How to execute script on schedule?
(3 answers)
Closed 1 year ago.
I have my python scripts which execute every weekday and refreshes tableau datasets. This is all being done incremental refresh on a daily basis.
Now, during weekends instead of manual refresh, I want these scripts to run with a full refresh. Any ideas on how we can achieve this?
Example:
I have tasks in my script like
#Run Tasks
refresh_tableA()
refresh_tableB()
refresh_tableC()
refresh_tablemain(i)
refresh_tableD()
Where the refresh_maintable() is an incremental refresh. Want that to trigger to a full refresh on weekends.
You need to have two cron entries.
week days only - call the python code and order it to create partial refresh
wekends - call the python code and order it to create full refresh
I think what you are looking for is a scheduler.
You have 2 options here either use Task Scheduler/Cronjob (Based on your OS) or using python library for this.
I think there is library for cronjob in python pycron.
That way you can schedule you scripts easily. let me know if you still dont understand the flow or want code. Most probably you'll find something on github.
This question already has answers here:
How do I run a Python script from C#?
(8 answers)
Run a python script from unity, to use its output (text file) in my game later
(4 answers)
Closed 1 year ago.
I want to make a Unity game that gives data to a subprogram written in Python and this subprogram gives back an answer, that is then processed in the C# game.
I don't really know how to approach this.
Can I run python code from C# somehow or do I let two separate programs running, that exchange data somehow?
Both programs also need access to the same database.
You can make a python server using flask or fastAPI and then access it from C#. I am not sure how to get that done with C#(I dont know C# :( ), but there must be some libraries to call servers. Also, you could save the database on the cloud such as mongoDB.
I am trying to figure out an elegant way to share a variable between a windows form app and a python script running in the background. The variable would be used solely to update a progress bar in the windows form based on the the long running process in the python script. More specifically, a windows timer will fire every n seconds, check the variable, then update the progress bar value. Sound stupid enough yet? I'll try to explain the need for this below.
I have a windows app that lets a user define a number of parameters to fire off a long running process (python script). Without getting into unnecessary detail, this long running process will insert many (100k+ records) into a sqlite database over a significant period of time. In order to make the python script as performant as possible, I don't call commit on the sqlite database until the very end of the python script. Trying to query the sqlite database from the windows app (via System.Data.Sqlite) before the commit occurs always yields 0 records, regardless of far along the process is.
The windows app will know how many total records will be inserted by the python process, so determining progress will be straight-forward enough, assuming I can get access to a record count in the python script.
I know I could do this with a text file, but is there any better way?
Easiest solution is probably to just have the python script print to stdout: say each time an item is processed, print a line with a number representing how many items have been processed (or a percentage). Then have the forms application read the output line by line, updating the progressbar based on that information.
If you use IronPython instead, you can create the Form-with-progress-bar in Python and manipulate it directly.
Alternately, your Winforms app can host the script and share variables.