Develop Reference

Is there a way to run azure-devops pipeline from python?

I have multiple pipelines with scripts defined on my azure devops.
I would like to be able to run them from my local machine using python.
Is this possible?
If yes, how to achieve it?
Regards,
Maciej

You can't run them in this way that you will take YAML code and put it python or actually any language and run it locally. You need to build agent to run your pipeline. So you can create a pool, install agent on you local machine, change your pipeline to use this pool.

Trigger an Azure Devops pipeline from an azure function using python

I have an existing Devops pipeline that trains ML models. To guaranty the robustness for the models, it will be necessary to retrain it periodically. For this I decided to create an Azure function that will be executed each month. will It collect the new data, apply the data pre-processing and finally trigger the Azure Devops training pipeline. All of this must be done with python.
By doing a research, I understood that this can be done using REST Azure Devops API request.
I founded this python git repo https://github.com/microsoft/azure-devops-python-api which provides an API to communicate with Azure DevOps. I executed the code provided by this package which displays the list of my devops projects. But I can't found how trigger the pipeline.
Assuming that my organisation named ORGA1, the project named PROJ1 and the pipeline that I want ti execute named PIPELINE1, How can I launch it using an Azure function or even a simple python script ?
PS: I am using a python 3.9 Timer Trigger Azure function.
Thank you in advance for your help.
EDIT
I tired to use LOGIC APP to do this like #mohammed described in the comment and I think that this is a good solution. Above the workflow that I created:
So I launch the logic app each X hours, this will trigger the azure Devops, and ASA it end training with Success it will send me an email.
I have one error here, is that I am creating a new release and not triggering a specific pipeline each time. But navigating in the different actions under the devops service, I cannot found any thing related to launching a devops pipeline. Can anyone have an idea how to do it.

You can use a Logic App with a timer to trigger your DevOps pipeline instead of an Azure function, as it has all the built-in connectors required to interface with your DevOps. See : https://www.serverlessnotes.com/docs/keep-your-team-connected-using-azure-devops-and-azure-logic-apps

Thanks to the tips provided by #Mohammed I found a solution. Logic App provides what I am looking for. Under the list of Devops connectors provided the by the Logic App, there is a connector named Queue a new build and this is exactly what I am looking for. This is my first experiment architecture and I will update it later by adding Azure Function service before calling the Devops Pipeline.

You may try using Azure Durable Functions, you can kind of replicate what a Logic App does while still using Azure Functions. See documentation here 1

AWS CLOUD9: How can I automate a task to be run on daily basis?

I've created a python script that grabs information from an API and sends it in an email. I'd like to automate this process so to run on daily basis let's say at 9AM.
The servers must be asleep when they will not be running this automation.
What is the easiest way to achieve this?
Note: Free version of AWS.

cloud9 is the ide that lets you write, run, and debug your code with just a browser.
"It preconfigures the development environment with all the SDKs, libraries, and plug-ins needed for serverless development. Cloud9 also provides an environment for locally testing and debugging AWS Lambda functions. This allows you to iterate on your code directly, saving you time and improving the quality of your code."
okay for the requirement you have posted :-
there are 2 ways of achieving this
on a local system use cron job scheduler daemon to run the script. a tutorial for cron.tutorial for cron
same thing can also be achieved by using a lambda function. lambda only runs when it is triggered, using compute resources for that particular time when it is invoked so your servers are sleeping for the rest of time( technically you are not provisioning any server for lambda)
convert your script in a function for lambda. and then use event bridge service where you can specify a corn expression to run your script everyday at 9am. wrote an article on the same may it can help.
note :- for email service you can use ses https://aws.amazon.com/ses/. my article uses ses.

To schedule Events you'd need a Lambda function with Cloudwatch events such as follow. https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/RunLambdaSchedule.html
Cloud9 is an IDE.

Azure devops for Python

I am trying to create an azure ci/cd pipeline for my python application. I have tried many ways but not get success. I can create CI successfully even in some cases cd also but not able to see the output on the azure app service.
I use Linux app services that use the Python 3.7 version.
I can create ci-cd successfully using the YAML file but I want to create using the classic editor without YAML, as I have some restrictions using yaml.

I will post the steps I deploy a simple hello world project with DevOps CI/CD pipeline.
1. Create pipeline:
2. Create Release pipeline:
3. Save and queue your pipeline, the release pipeline would be triggered. Here is the file structure on Azure KUDU:

How to boot up a test pubsub emulator from python for automated testing

I'm working on a flask API, which one of the endpoint is to receive a message and publish it to PubSub. Currently, in order to test that endpoint, I will have to manually spin-up a PubSub emulator from the command line, and keep it running during the test. It working just fine, but it wouldn't be ideal for automated test.
I wonder if anyone knows a way to spin-up a test PubSub emulator from python? Or if anyone has a better solution for testing such an API?

As far as I know, there is no Python native Google Cloud PubSub emulator available.
You have few options, all of them require launching an external program from Python:
Just invoke the gcloud command you mentioned: gcloud beta emulators pubsub start [options] directly from your python application to start this as an external program.
The PubSub emulator which comes as part of Cloud SDK is a JAR file bootstrapped by the bash script present in CLOUD_SDK_INSTALL_DIR/platform/pubsub-emulator/bin/cloud-pubsub-emulator. You could possibly run this bash script directly.
Here is a StackOverflow answer which covers multiple ways to launch an external program from Python.
Also, it is not quite clear from your question how you're calling the PubSub APIs in Python.
For unit tests, you could consider setting up a wrapper over the code which actually invokes the Cloud PubSub APIs, and inject a fake for this API wrapper. This way, you can test the rest of the code which invokes just your fake API wrapper instead of the real API wrapper and not worry about starting any external programs.
For integration tests, the PubSub emulator will definitely be useful.

This is how I usually do:
1. I create a python client class which does publish and subscribe with the topic, project and subscription used in emulator.
Note: You need to set PUBSUB_EMULATOR_HOST=localhost:8085 as env in your python project.
2. I spin up a pubsub-emulator as a docker container.
Note: You need to set some envs, mount volumes and expose port 8085.
set following envs for container:
PUBSUB_EMULATOR_HOST
PUBSUB_PROJECT_ID
PUBSUB_TOPIC_ID
PUBSUB_SUBSCRIPTION_ID
Write whatever integration tests you want to. Use publisher or subscriber from client depending on your test requirements.