Develop Reference

office365 sharepoint open binary gives missing schema error

I am trying to read a file from sharepoint using the office 365 package. My code looks like this:
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.file import File
app_settings = {
'url': 'https://xxxx.sharepoint.com/sites/mysite/',
'client_id': 'id',
'client_secret': 'secret'}
context_auth = AuthenticationContext(url=app_settings['url'])
context_auth.acquire_token_for_app(client_id=app_settings['client_id'],
client_secret=app_settings['client_secret'])
ctx = ClientContext(app_settings['url'], context_auth)
File.open_binary(ctx, '/sites/mysite/Documents/Dictionary.csv')
I've tried several variations of the server relative url and get the error every time:
MissingSchema: Invalid URL "<bound method ClientContext.service_root_url of
<office365.sharepoint.client_context.ClientContext object at
0x00019D2CCED68>>web/getfilebyserverrelativeurl
('/sites/mysite/Documents/Dictionary.csv')/\\$value": No schema supplied.
Perhaps you meant http://<bound method ClientContext.service_root_url of
<office365.sharepoint.client_context.ClientContext object at
0x00019D2CCED68>>web/getfilebyserverrelativeurl('/sites/mysite/Documents/Dictionary.csv')/\$value?
Is there a simple way to read and download files from sharepoint online using the office365 library?

File Download from Sharepoint using Python

I am trying to download an excel file from Sharepoint with the office365 module. Here's my code.
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.file import File
app_settings = {
'url': 'https://xxxxx/sites/DownloadFiles',
'client_id': 'xxxxxx',
'client_secret': 'xxxxxx',
}
if name == 'main':
ctx_auth = AuthenticationContext(url=app_settings['url'])
ctx_auth.acquire_token_for_app(client_id=app_settings['client_id'],
client_secret=app_settings['client_secret'])
ctx = ClientContext(app_settings['url'], ctx_auth)
path = "F:\myexcel.xlsx"
response = File.open_binary(ctx, "/Shared%20Documents/myexcel.xlsx")
response.raise_for_status()
with open(path, "wb") as local_file:
local_file.write(response.content)
When I run that code, I get the following error:
400 Client Error: Bad Request for url: https://xxx/DownloadFiles/_api/web/getfilebyserverrelativeurl('/Shared%20Documents/myexcel.xlsx')/%5C$value

I am able to reproduce the same issue on my SPO.
Please modify the code as below to fix it:
response = File.open_binary(ctx, "/sites/{abc}/Shared%20Documents/source.txt")
Such as i want to download file from a site collection like "https://xxxx.sharepoint.com/sites/abc", the serverrelativeurl is "/sites/abc"
You can get the serverrelativeurl via '_spPageContextInfo' object:
And there is another SharePoint library 'shareplum' that provided some easy ways to operate files, you may have a try.
Best Regards,
Baker Kong

Trying to query an API from AWS Lambda

I'm trying to query an open source API that returns IP geolocation information by sending a GET request with the IP.
I'm testing the code with a key that contains an IP address (located in key1). I'm trying to fetch the information after the request is sent but I'm not sure what I'm doing wrong.
I have tried appending the IP to the end of the url (as the geoip API instructs) but I keep getting syntax errors.
import json
from botocore.vendored import requests
def lambda_handler(resp, requests, event):
event = event.key1
url = "https://freegeoip.app/json/" +event
headers = {
'accept': "application/json",
'content-type': "application/json"
}
response = requests.request("GET", url, headers=headers)
print(response.text)
I have the code working in regular python syntax below, just don't know how to get it working with lambda
import requests
userIP = '54.81.183.174'
def theFunction():
url = "https://freegeoip.app/json/" + userIP
headers = {
'accept': "application/json",
'content-type': "application/json"
}
response = requests.request("GET", url, headers=headers)
print(response.text)
theFunction()

Your code is using the requests module, which is not installed with AWS Lambda.
You can package it for use with an AWS Lambda function (see python - Cannot use Requests-Module on AWS Lambda - Stack Overflow), but it is simpler to use urllib, which is part of standard Python3.
Here is some code that works:
import urllib.request
import json
def lambda_handler(event, context):
ip = event['ip']
with urllib.request.urlopen("https://freegeoip.app/json/" + ip) as f:
data = json.loads(f.read())
print(data)
print(data['city'])
You can trigger it with test data:
{
"ip": "54.81.183.174"
}

Problem authenticating to Power BI REST API with Python

I've created a push streaming dataset (history on) and I've managed to post data to it from a Python script using the "Push URL" which I got from the API Info tab for the dataset in question. What I also need to do is to delete the historic data so as to clear out my test data and/or be able to reset the dataset and re-populate from scratch as and when necessary.
The Push Url is of the form https://api.powerbi.com/beta/xxxxxxxx/datasets/xxxxxxxxxxxx/rows?key=xxxxxxxxxxxxxxx
The following code works fine and the data is posted;
import requests
import pyodbc as db
import pandas as pd
API_ENDPOINT = "https://api.powerbi.com/beta/xxxxxxxx/datasets/xxxxxxxxxxxx/rows?key=xxxxxxxxxxxxxxx"
dbcon = db.connect('DRIVER={SQL Server};SERVER=tcp:fxdb.database.windows.net;DATABASE=FXDatabase;UID=xxxx;PWD=xxxx')
df = pd.read_sql("select statement etc...", dbcon)
data = df.to_dict(orient='records')
response = requests.post(API_ENDPOINT, json=data)
But adding this:
response = requests.delete(API_ENDPOINT)
gives me:
404
{
"error":{
"code":"","message":"No HTTP resource was found that matches the request URI 'http://api.powerbi.com/beta/...
I couldn't figure this out so I started looking into OAuth2 authentication thinking that perhaps the Auth URL is only for posting data. After registering the app at https://dev.powerbi.com/apps my code now looks like this:
import requests
import pyodbc as db
import pandas as pd
API_ENDPOINT = "https://api.powerbi.com/beta/xxxxxxxxxxxxxx/datasets/xxxxxxxxxxxxxxx/rows"
data = {
'grant_type': 'password',
'scope': 'openid',
'resource': r'https://analysis.windows.net/powerbi/api',
'client_id': 'xxxxxxxxx',
'username': 'xxxxxxxxx',
'password': 'xxxxxxxx'
}
response = requests.post('https://login.microsoftonline.com/common/oauth2/token', data=data)
access_token = response.json().get('access_token')
headers = {'Authorization': 'Bearer ' + access_token}
dbcon = db.connect('DRIVER={SQL Server};SERVER=tcp:fxdb.database.windows.net;DATABASE=FXDatabase;UID=xxxx;PWD=xxxx')
df = pd.read_sql("select statement etc...", dbcon)
data = df.to_dict(orient='records')
response = requests.post(API_ENDPOINT, json=data, headers=headers)
response = requests.delete(API_ENDPOINT, headers=headers)
The authentication works, returning status code 200. The POST returns 401 (this worked with the previous method) and the DELETE still returns 404.

Thanks to jonrsharpe who pointed me in the right direction.
Revisiting the API documentation I discovered a call to get the table names;
GET https://api.powerbi.com/v1.0/myorg/datasets/{datasetKey}/tables
so after authenticating I ran;
response = requests.get("https://api.powerbi.com/v1.0/myorg/datasets/xxxxxxxx/tables", headers=headers)
The content of the response told me that there was a table called "RealTimeData" inside my dataset, must be a default name because I haven't knowingly created this table.
I have now updated the endpoint to;
API_ENDPOINT = "https://api.powerbi.com/v1.0/myorg/datasets/xxxxxxxxx/tables/RealTimeData/rows"
and all works perfectly.
Thanks Jon!

Azure Access Token Request returning HTML

I am trying to request an authorization code as documented here.
I am using Python requests package to do this and have the following example code:
import requests
auth_endpoint = 'https://login.microsoftonline.com/%s/oauth2/authorize?api-version=1.0' % TENANT_ID
payload = {
'client_id': CLIENT_ID,
'response_type': 'code',
'resource': APP_ID_URI,
'redirect_uri': REPLY_URL
}
response = requests.get(url=auth_endpoint, data=payload)
However, when I run the code above, I get back HTML in the body and not the response I'm expecting. It seems like the HTML code is for a login page.
When I take the formatted endpoint URI and plug it into a browser, I am able to get the auth code from the redirect URI. But, is there a way to get this from the body of the response while still using the requests package?

Please use session class of requests module to implement your requirement. Please refer to the following code sample:
import requests
s = requests.Session()
USERNAME = '<username_email>'
PASSWORD = '<userpassword>'
s.auth = (USERNAME, PASSWORD)
TENANT_ID = '<tenant_id>'
# Authorize URL
authorize_url = 'https://login.microsoftonline.com/%s/oauth2/authorize' % TENANT_ID
# Token endpoint.
token_url = 'https://login.microsoftonline.com/%s/oauth2/token' % TENANT_ID
payload = { 'response_type': 'code',
'client_id': '<tenant_id>',
'redirect_uri': 'http://localhost',
'authority' :'authority'
}
response = s.get(authorize_url, params=payload ,allow_redirects=True)
print response
print response.url
Any further concern, please feel free to let me know.