Develop Reference

How to create a new branch, push a text file and send merge request to a gitlab repository using Python?

I found
https://github.com/python-gitlab/python-gitlab, but I was unable to understand the examples in the doc.

That's right there are no tests we can find in the doc. Here's a basic answer for your question.
If you would like a complete working script, I have attached it here:
https://github.com/torpidsnake/common_scripts/blob/main/automation_to_create_push_merge_in_gitlab/usecase_gitlab_python.py
Breaking down the steps below:
Create an authkey for you: Follow the steps here: https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
Create a gitlab server instance of your project
server = gitlab.Gitlab('https://gitlab.example.com', private_token=YOUR_API_TOKEN)
project = server.projects.get(PROJECT_ID)
Create a branch using:
branch = project.branches.create(
{"branch": branch_name, "ref": project.default_branch}
)
Upload a file using:
project.files.create(
{
"file_path": file_name,
"branch": branch.name,
"content": "data to be written",
"encoding": "text", # or 'base64'; useful for binary files
"author_email": AUTHOR_EMAIL, # Optional
"author_name": AUTHOR_NAME, # Optional
"commit_message": "Create file",
}
)
Create a merge request using:
project.mergerequests.create(
{
"source_branch": branch.name,
"target_branch": project.default_branch,
"title": "merge request title",
}
)

Looking at python-gitlab, I don't see some of the things you are looking for. In that case, I suggest you break it apart and do the individual steps using more basic tools and libraries.
The first two parts you don't need to use Gitlab API to do. You can basically use Python to do the clone, branch, edit, and commit calls using git.exe and against your disk. In some ways, that is easier since you can duplicate the calls yourself. You could use GitPython.
I'd recommend you do that through one of these methods instead of trying to do it via Gitlab API. It's easier to understand, debug, and investigate if you do the branch work locally (or even inside a CI).
Once you push up the code into a branch, you can use Gitlab's API to create a merge request via REST (such as the requests library). The description for creating the MR is at https://docs.gitlab.com/ee/api/merge_requests.html#create-mr and most of the fields are optional so the minimum looks like:
{
"id": "some-user%2Fsome-project",
"source_branch": "name_of_your_mr_branch",
"target_branch": "main",
"title": "Automated Merge Request..."
}
This is an authenticated POST call (to create). Between those links, you should have most of what you need to do this.

How to update Chart Title via Google Sheet API (python)

I have a google chart whose ID I have already extracted via .get method.
I now need to change the title of the chart (and nothing else). The charts come from external template, so I do not know anything about them, other than the name.
I tried the following request:
body = {
"requests": [
{
"updateChartSpec": {
"chartId": 1944564251,
"spec": {
"title": "NEW NAME OF CHART"}
}}]
}
r = sheet.batchUpdate(spreadsheetId=DESTINATION_SPREADSHEET_ID, body=body).execute()
but I am getting this error:
"Invalid requests[0].updateChartSpec: One of basicChart, pieChart, bubbleChart, candelstickChart, histogramChart, or orgChart must be set on chartSpec."
I don't want to change anything in the chart other than the title. How can I go about it?

I believe your goal and your current situation as follows.
You want to modify the title of the chart on Google Spreadsheet.
You want to achieve this using googleapis for Python.
You have already been able to get and put values for Google Spreadsheet using Sheets API.
Modification points:
In the current stage, it seems that UpdateChartSpecRequest of Sheets API cannot still update the chart title using only the property of title. I thought that the reason of your issue is due to this. This has already been reported to the issue tracker as a Feature Request. Ref
I think that if fields is added to this request, your script might work. But, in the current stage, it is required to use the workaround.
The workaround is as follows.
Retrieve the chart object using the chart ID and spreadsheets.get method in Sheets API.
Modify the chart object.
Using UpdateChartSpecRequest, the modified chart object is used. By this, the chart is updated.
I have already answered this workaround at here. But, this is for Google Apps Script. When I searched this workaround using googleapis for Python, I couldn't find it. So I answerd this using googleapis for Python.
When above workaround is reflected to the script of Python, it becomes as follows.
Modified script:
DESTINATION_SPREADSHEET_ID = "###" # Please set the Spreadsheet ID.
chartId = 1944564251 # Please set the chart ID.
newTitle = "NEW NAME OF CHART"
service = build('sheets', 'v4', credentials=creds)
res = service.spreadsheets().get(spreadsheetId=DESTINATION_SPREADSHEET_ID, fields='sheets(charts)').execute()
chart = None
for s in res.get('sheets'):
charts = s.get('charts')
if charts:
for c in charts:
if c.get('chartId') == chartId:
chart = c
if chart:
chart.pop('position')
chart['spec']['title'] = newTitle
body = {"requests": [{"updateChartSpec": chart}]}
r = service.spreadsheets().batchUpdate(spreadsheetId=DESTINATION_SPREADSHEET_ID, body=body).execute()
print(r)
else:
print("Chart cannot be found.")
References:
Method: spreadsheets.get
Method: spreadsheets.batchUpdate
UpdateChartSpecRequest
Issue tracker.
Add fields property to UpdateChartSpecRequest of batchUpdate method in Sheets API
Added:
From your following replying,
Thank you for your response. I tried the code above and getting this error: Invalid requests[0].updateChartSpec: chartSpec.basicChart.lineSmoothing not supported when chartSpec.basicChart.chartType is COMBO
I could replicate your error. I confirmed that in the current stage, it seems that "lineSmoothing": true cannot be used with COMBO chart using UpdateChartSpecRequest. So in the current stage, when Sheets API is used, it is required to remove the property of "lineSmoothing": true as follows. In this case, please modify above sample script as follows.
From:
if chart:
chart.pop('position')
To:
if chart:
chart.pop('position')
if chart['spec'].get('basicChart') and chart['spec']['basicChart']['chartType'] == 'COMBO' and chart['spec']['basicChart'].get('lineSmoothing') and chart['spec']['basicChart']['lineSmoothing'] == True:
chart['spec']['basicChart'].pop('lineSmoothing')
By above modification, the error can be removed. But in this case, unfortunately, the chart is changed. Because "lineSmoothing": true is not used. So, in this case, how about reporting this issue as the issue tracker as the future request? Ref
So, as a workaround, how about the following method? In this workaround, Web Apps created by Google Apps Script is used as the wrapper API. When Google Apps Script is used, the chart title can be directly modified.
Usage:
1. Create new project of Google Apps Script.
Sample script of Web Apps is a Google Apps Script. So please create a project of Google Apps Script.
If you want to directly create it, please access to https://script.new/. In this case, if you are not logged in Google, the log in screen is opened. So please log in to Google. By this, the script editor of Google Apps Script is opened.
2. Sample script.
Please copy and paste the following script to the created Google Apps Script project and save it.
function doGet(e) {
const {spreadsheetId, chartId, newTitle} = e.parameter;
const sheets = SpreadsheetApp.openById(spreadsheetId).getSheets();
let done = false;
for (let i = 0; i < sheets.length; i++) {
const charts = sheets[i].getCharts();
for (let j = 0; j < charts.length; j++) {
if (charts[j].getChartId() == chartId) {
const chart = charts[j].modify().setOption("title", newTitle).build();
sheets[i].updateChart(chart);
done = true;
break;
}
}
if (done) break;
}
return ContentService.createTextOutput("Done.");
}
3. Deploy Web Apps.
The detail information can be seen at the official document.
On the script editor, at the top right of the script editor, please click "click Deploy" -> "New deployment".
Please click "Select type" -> "Web App".
Please input the information about the Web App in the fields under "Deployment configuration".
Please select "Me" for "Execute as".
This is the important of this workaround.
Please select "Anyone" for "Who has access".
In this case, the user is not required to use the access token. So please use this as a test case.
Of course, you can also access to your Web Apps using the access token. Please check this report.
Please click "Deploy" button.
Copy the URL of Web App. It's like https://script.google.com/macros/s/###/exec.
When you modified the Google Apps Script, please modify the deployment as new version. By this, the modified script is reflected to Web Apps. Please be careful this.
You can see the detail of this at the report of "Redeploying Web Apps without Changing URL of Web Apps for new IDE".
4. Testing.
When above Web Apps is tested, the sample python script is as follows. Please set your URL of Web Apps.
import requests
url = "https://script.google.com/macros/s/###/exec"
url += "?spreadsheetId=###&chartId=###&newTitle=###"
res = requests.get(url)
print(res.text)
When an error occurs, please confirm the settings of Web Apps again.
References:
Web Apps
Taking advantage of Web Apps with Google Apps Script

Get OneNote timestamps through the API

Is there a way to get the timestamps for content in a OneNote page through the OneNote API, or another way? I have been using the API to get the html content and metadata of my pages, however I have not found a way to get the timestamps. I want to get these so I can see an edit history. These exist in OneNote as they show up when you highlight text and right click on it, as shown: OneNote Timestamp. Basically is there anyway to export this information? Ideally this would be in Python, but I'm not picky.
Thanks.

Assuming you are looking for the edit history of a page, it seems like your best bet is to use the "lastModifiedDateTime" field in Microsoft's responses to GET /me/onenote/pages/{id}.
Here is a sample response:
HTTP/1.1 200 OK
Content-type: application/json
Content-length: 312
{
"title": "title-value",
"createdByAppId": "createdByAppId-value",
"links": {
"oneNoteClientUrl": {
"href": "href-value"
},
"oneNoteWebUrl": {
"href": "href-value"
}
},
"contentUrl": "contentUrl-value",
"content": "content-value",
"lastModifiedDateTime": "2016-10-19T10:37:00Z"
}
You can find the full details in their OneNote REST API documentation at this link: https://learn.microsoft.com/en-us/graph/api/page-get?view=graph-rest-1.0.
OneNote notebooks and sections also have a "lastModifiedDateTime".
You can use Python to parse the response for the "lastModifiedDateTime" field. See the two pages below for guidance on parsing API responses with Python.
Selecting fields from JSON output
What's the best way to parse a JSON response from the requests library?
Best of luck with the rest of your project!

Youtube API get the current search term?

I am trying to get a JSON List of Videos. My Problem is that I can´t get the current Searchresult. If I go on youtube I get different Searchresult as if I run my python Script. I ask because I recognize that there is no such term handled in Stackoverflow.
Code:
def getVideo():
parameters = {"part": "snippet",
"maxResults": 5,
"order": "relevance",
"pageToken": "",
"publishedAfter": None,
"publishedBefore": None,
"q": "",
"key": api_key,
"type": "video",
}
url = "https://www.googleapis.com/youtube/v3/search"
parameters["q"] = "Shroud"
page = requests.request(method="get", url=url, params=parameters)
j_results = json.loads(page.text)
print(page.text)
#print(j_results)
getVideo()
I have some thoughts. I think its because of the variables publishedAfter and publishedbefore but I dont know how I can fix it.
Best Regards
Cren1993

Search: list
Returns a collection of search results that match the query parameters specified in the API request. By default, a search result set identifies matching video, channel, and playlist resources, but you can also configure queries to only retrieve a specific type of resource.
No were in there does it mention that the search results for the YouTube api will return the exact same results as the YouTube Website. They are different systems. The API is for use by third party developers like yourself. The YouTube website is controlled by Google and probably contained extra search features that google has not exposed to third party developers either because they cant or they dont want to.

Listing extractors from import.io

I would like to know how to get the crawling data (list of URLs manually input through the GUI) from my import.io extractors.
The API documentation is very scarce and it does not specify if the GET requests I make actually start a crawler (and consume one of my crawler available runs) or just query the result of manually launched crawlers.
Also I would like to know how to obtain the connector ID, as I understand, an extractor is nothing more than a specialized connector, but when I use the extractor_id as the connector id for querying the API, I get the connector does not exist.
A way I thought I could have listed the URLs I have in one off my extractors is this:
https://api.import.io/store/connector/_search?
_sortDirection=DESC&_default_operator=OR&_mine=true&_apikey=123...
But the only result I get is:
{ "took": 2, "timed_out": false, "hits": {
"total": 0,
"hits": [],
"max_score": 0 } }
Nevertheless, even if I would get a more complete response, the example result I see in the documentation ddoes not mention any kind of list or element containing the URLs I'm trying to get from my import.io account.
I am using python to create this API

The legacy API will not work for any non-legacy connectors, so you will have to use the new Web Extractor API. Unfortunately, there is no documentation for this.
Luckily, with some snooping you can find the following call to list connectors connected to your apikey:
https://store.import.io/store/extractor/_search?_apikey=YOUR_API_KEY
From here, You check each hit and verify the _type property is set to EXTRACTOR. This will give you access to, among other things, the GUID associated with the extractor and the name you chose for it when you created it.
You can then do the following to download the latest run from the extractor in CSV format:
https://data.import.io/extractor/{{GUID}}/csv/latest?_apikey=YOUR_API_KEY
This was found in the Integrations tab of every Web Extractor. There are other queries there as well.
Hope this helps.