I'm using the code below to pull specific tweets and save to a .json file format and keep getting the following error:
IOError: [Errno 2] No such file or directory 'data/greenroof.json'
The code in question:
import json
import tweepy
API_KEY = "key_here"
API_SECRET = "secret here"
TOKEN_KEY = "t_key_here"
TOKEN_SECRET = "t_secret_here"
auth = tweepy.OAuthHandler(API_KEY, API_SECRET)
auth.set_access_token(TOKEN_KEY, TOKEN_SECRET)
api = tweepy.API(auth)
query = '#greenroof'
cursor = tweepy.Cursor(api.search, q=query, lang="en")
for page in cursor.pages():
tweets = []
for item in page:
tweets.append(item._json)
with open('data/greenroof.json', 'wb') as outfile:
json.dump(tweets, outfile)
Please forgive the spacing of the above code, I cant appear to get it to present accurately.
I've used the mkdir data request on the command line, ensuring the path for said directory matches the location of script.
The error suggest that the file doesn't exist.
Any help or pointers greatly appreciated.
Oh, and go easy on me folks, total newbie to code & python.
if the data directory is in your current working directory, make sure to add a ./ to the beginning of the path, signifying the current working directory.
i.e.
with open('./data/greenroof.json', 'wb') as outfile:
As the title says, I have access to a shared folder where some files are uploaded. I just want to donwload an specific file, called "db.dta". So, I have this script:
def download_file(url, filename):
url = url
file_name = filename
with open(file_name, "wb") as f:
print("Downloading %s" % file_name)
response = requests.get(url, stream=True)
total_length = response.headers.get('content-length')
if total_length is None: # no content length header
f.write(response.content)
else:
dl = 0
total_length = int(total_length)
for data in response.iter_content(chunk_size=4096):
dl += len(data)
f.write(data)
done = int(50 * dl / total_length)
sys.stdout.write("\r[%s%s]" % ('=' * done, ' ' * (50-done)) )
sys.stdout.flush()
print(" ")
print('Descarga existosa.')
It actually download shares links of files if I modify the dl=0 to 1, like this:
https://www.dropbox.com/s/ajklhfalsdfl/db_test.dta?dl=1
The thing is, I dont have the share link of this particular file in this shared folder, so if I use the url of the file preview, I get an error of denied access (even if I change dl=0 to 1).
https://www.dropbox.com/sh/a630ksuyrtw33yo/LKExc-MKDKIIWJMLKFJ?dl=1&preview=db.dta
Error given:
dropbox.exceptions.ApiError: ApiError('22eaf5ee05614d2d9726b948f59a9ec7', GetSharedLinkFileError('shared_link_access_denied', None))
Is there a way to download this file?
If you have the shared link to the parent folder and not the specific file you want, you can use the /2/sharing/get_shared_link_file endpoint to download just the specific file.
In the Dropbox API v2 Python SDK, that's the sharing_get_shared_link_file method (or sharing_get_shared_link_file_to_file). Based on the error output you shared, it looks like you are already using that (though not in the particular code snippet you posted).
Using that would look like this:
import dropbox
dbx = dropbox.Dropbox(ACCESS_TOKEN)
folder_shared_link = "https://www.dropbox.com/sh/a630ksuyrtw33yo/LKExc-MKDKIIWJMLKFJ"
file_relative_path = "/db.dat"
res = dbx.sharing_get_shared_link_file(url=folder_shared_link, path=file_relative_path)
print("Metadata: %s" % res[0])
print("File data: %s bytes" % len(res[1].content))
(You mentioned both "db.dat" and "db.dta" in your question. Make sure you use whichever is actually correct.)
Additionally, note if you using a Dropbox API app registered with the "app folder" access type: there's currently a bug that can cause this shared_link_access_denied error when using this method with an access token for an app folder app.
I need to get the last modified date of a document in the SharePoint using SharePoint rest API for python
Basically I want to get the most recently modified file from a SharePoint site I want to get the last modified date to verify if the current document is the latest
I am trying to find if there is any method which can help get the last modified date of documents.
I suggest using O365 Python package.
https://github.com/O365/python-o365
Below example shows how you can print modified time and then download the file. In this example I am downloading excel files.
Note: I have all my passwords and etc stored in a dictionary within a settings file I created hence why you see me importing that file. This is not needed and is a personal preference.
from O365 import Account
from settings import settings
save_path = "\path\to\save\file\\"
# user log in information using client credentials and client id for Microsoft Graph API
credentials = (settings['client_credentials']['client_id'],
settings['client_credentials']['client_secret'])
# the default protocol will be Microsoft Graph
account = Account(credentials, auth_flow_type='credentials',
tenant_id=settings['client_credentials'
]['client_tenant_id'])
if account.authenticate():
print('Authenticated!')
# initiate share_point account
share_point = account.sharepoint()
# change this site id to whatever your SharePoint site id is
site_id = 'your site id'
# get the site
site = share_point.get_site(site_id)
# create drive item
my_drive = site.get_default_document_library()
# navigate to root folder of the drive
root_folder = my_drive.get_root_folder()
# this will get all the folders under your root folder
child_folders = root_folder.get_child_folders()
for folder in child_folders:
if folder.name == 'Your SharePoint folder name':
for item in folder.get_items():
try:
if item.name.lower().endswith(('.xls', '.xlsx')):
# print last modified date
print(item.modified)
# download files
item.download(save_path)
except Exception, e:
print('File not found!')
print(e)
I have written a python script that scrapes account numbers from a pdf document and it works great locally. I've used pyinstaller to extract the script into an executable and saved it onto a flash drive. I moved the pdf document into what I thought was the CWD but when I try to run it, my terminal tells me the pdf I'm tyring to open cannot be found.
Eventually I want my client to be able to put the flash drive in, move the newest version of the pdf over to the same directory, and run the script. I need a nudge in the right direction as how to specify the file path for PyPDF2. Here is my code so far.
def getDataFromPdf():
acctNumberRegex = re.compile(r'\d\d\d\d\d-\d\d\d-\d\d\d\d')
pdfFile = open('records.pdf', 'rb')
reader = PyPDF2.PdfFileReader(pdfFile)
for pageNum in range(0,10):
page = reader.getPage(pageNum).extractText()
accounts = acctNumberRegex.findall(page)
for acct in accounts:
if acct not in results:
results.append(acct)
print(len(results))
Thanks!
You could pass the file path as an argument into your function.
So:
def getDataFromPdf(filePath):
acctNumberRegex = re.compile(r'\d\d\d\d\d-\d\d\d-\d\d\d\d')
pdfFile = open(filePath + 'records.pdf', 'rb')
reader = PyPDF2.PdfFileReader(pdfFile)
for pageNum in range(0,10):
page = reader.getPage(pageNum).extractText()
accounts = acctNumberRegex.findall(page)
for acct in accounts:
if acct not in results:
results.append(acct)
print(len(results))
Solution via r/learnpython
SCRIPT_DIR = Path(sys.executable).parent
def getDataFromPdf():
pdf_file = SCRIPT_DIR / 'records.pdf'
print(pdf_file.resolve())
with open(pdf_file.resolve(),'rb') as records:
acctNumberRegex = re.compile(r'\d\d\d\d\d-\d\d\d-\d\d\d\d')
reader = PyPDF2.PdfFileReader(records)
for pageNum in range(0,reader.numPages):
page = reader.getPage(pageNum).extractText()
accounts = acctNumberRegex.findall(page)
for acct in accounts:
if acct not in results:
results.append(acct)
print(str(len(results)) + " account numbers pulled from PDF")
I want to upload a file from my python script to my dropbox account automatically. I can't find anyway to do this with just a user/pass. Everything I see in the Dropbox SDK is related to an app having user interaction. I just want to do something like this:
https://api-content.dropbox.com/1/files_put//?user=me&pass=blah
The answer of #Christina is based on Dropbox APP v1, which is deprecated now and will be turned off on 6/28/2017. (Refer to here for more information.)
APP v2 is launched in November, 2015 which is simpler, more consistent, and more comprehensive.
Here is the source code with APP v2.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import dropbox
class TransferData:
def __init__(self, access_token):
self.access_token = access_token
def upload_file(self, file_from, file_to):
"""upload a file to Dropbox using API v2
"""
dbx = dropbox.Dropbox(self.access_token)
with open(file_from, 'rb') as f:
dbx.files_upload(f.read(), file_to)
def main():
access_token = '******'
transferData = TransferData(access_token)
file_from = 'test.txt'
file_to = '/test_dropbox/test.txt' # The full path to upload the file to, including the file name
# API v2
transferData.upload_file(file_from, file_to)
if __name__ == '__main__':
main()
The source code is hosted on GitHub, here.
Important Note: this answer is deprecated since dropbox uses v2 API now.
See the answer of #SparkAndShine for current API version solution
Thanks to #smarx for the answer above! I just wanted to clarify for anyone else trying to do this.
Make sure you install the dropbox module first of course, pip install dropbox.
Create an app under your own dropbox account in the "App Console". (https://www.dropbox.com/developers/apps)
Just for the record I created my App with the following:
a. App Type as "Dropbox API APP".
b. Type of data access as "Files & Datastores"
c. Folder access as "My app needs access to files already on Dropbox". (ie: Permission Type as "Full Dropbox".)
Then click the "generate access token" button and cut/paste into the python example below in place of <auth_token>:
import dropbox
client = dropbox.client.DropboxClient(<auth_token>)
print 'linked account: ', client.account_info()
f = open('working-draft.txt', 'rb')
response = client.put_file('/magnum-opus.txt', f)
print 'uploaded: ', response
folder_metadata = client.metadata('/')
print 'metadata: ', folder_metadata
f, metadata = client.get_file_and_metadata('/magnum-opus.txt')
out = open('magnum-opus.txt', 'wb')
out.write(f.read())
out.close()
print metadata
Here's my approach using API v2 (and Python 3). I wanted to upload a file and create a share link for it, which I could email to users. It's based on sparkandshine's example. Note I think the current API documentation has a small error which sparkandshine has corrected.
import pathlib
import dropbox
import re
# the source file
folder = pathlib.Path(".") # located in this folder
filename = "test.txt" # file name
filepath = folder / filename # path object, defining the file
# target location in Dropbox
target = "/Temp/" # the target folder
targetfile = target + filename # the target path and file name
# Create a dropbox object using an API v2 key
d = dropbox.Dropbox(your_api_access_token)
# open the file and upload it
with filepath.open("rb") as f:
# upload gives you metadata about the file
# we want to overwite any previous version of the file
meta = d.files_upload(f.read(), targetfile, mode=dropbox.files.WriteMode("overwrite"))
# create a shared link
link = d.sharing_create_shared_link(targetfile)
# url which can be shared
url = link.url
# link which directly downloads by replacing ?dl=0 with ?dl=1
dl_url = re.sub(r"\?dl\=0", "?dl=1", url)
print (dl_url)
import dropbox
access_token = '************************'
file_from = 'index.jpeg' //local file path
file_to = '/Siva/index.jpeg' // dropbox path
def upload_file(file_from, file_to):
dbx = dropbox.Dropbox(access_token)
f = open(file_from, 'rb')
dbx.files_upload(f.read(), file_to)
upload_file(file_from,file_to)
The only way to authenticate calls to the Dropbox API is to use OAuth, which involves the user giving permission to your app. We don't allow third-party apps to handle user credentials (username and password).
If this is just for your account, note that you can easily get an OAuth token for your own account and just use that. See https://www.dropbox.com/developers/blog/94/generate-an-access-token-for-your-own-account.
If this is for other users, they'll need to authorize your app once via the browser for you to get an OAuth token. Once you have the token, you can keep using it, though, so each user should only have to do this once.
Sorry if im missing something but cant you just download the dropbox application for your OS and then save the file (in windows) in:
C:\Users\<UserName>\Dropbox\<FileName>
i just ceated a python program to save a text file, checked my dropbox and it saves them fine.
For Dropbox Business API below python code helps uploading files to dropbox.
def dropbox_file_upload(access_token,dropbox_file_path,local_file_name):
'''
The function upload file to dropbox.
Parameters:
access_token(str): Access token to authinticate dropbox
dropbox_file_path(str): dropboth file path along with file name
Eg: '/ab/Input/f_name.xlsx'
local_file_name(str): local file name with path from where file needs to be uploaded
Eg: 'f_name.xlsx' # if working directory
Returns:
Boolean:
True on successful upload
False on unsuccessful upload
'''
try:
dbx = dropbox.DropboxTeam(access_token)
# get the team member id for common user
members = dbx.team_members_list()
for i in range(0,len(members.members)):
if members.members[i].profile.name.display_name == logged_in_user:
member_id = members.members[i].profile.team_member_id
break
# connect to dropbox with member id
dbx = dropbox.DropboxTeam(access_token).as_user(member_id)
# upload local file to dropbox
f = open(local_file_name, 'rb')
dbx.files_upload(f.read(),dropbox_file_path)
return True
except Exception as e:
print(e)
return False
If you need to upload a BIG file, you need to break up the file into chunks and upload the chunks one by one as follows. Inspired by this great medium artcle:
def upload_a_big_file(local_file_path: str, remote_file_path: str):
# grab your authenticated client
dbx = get_dropbox_client()
file_size = os.path.getsize(local_file_path)
# Upload 8 MB chunks at a time
CHUNK_SIZE = 8 * 1024 * 1024
with open(local_file_path, 'rb') as local_file:
uploaded_size = 0
upload_session_start_result = dbx.files_upload_session_start(local_file.read(CHUNK_SIZE))
cursor = dropbox.files.UploadSessionCursor(
session_id=upload_session_start_result.session_id,
offset=local_file.tell()
)
commit = dropbox.files.CommitInfo(
path=remote_file_path,
mode=dropbox.files.WriteMode.overwrite
)
print("Starting Upload.")
while local_file.tell() <= file_size:
if ((file_size - local_file.tell()) <= CHUNK_SIZE):
# Last chunk remaining, so commit
dbx.files_upload_session_finish(
local_file.read(CHUNK_SIZE),
cursor,
commit
)
print("Done uploading !")
break
else:
dbx.files_upload_session_append_v2(
local_file.read(CHUNK_SIZE),
cursor
)
cursor.offset = local_file.tell()
uploaded_size += CHUNK_SIZE
uploaded_percent = 100*uploaded_size/file_size
print('Uploaded {:.2f}%'.format(uploaded_percent))
Here is the code for uploading livevideo on dropbox using python in windows.
Hope this will help you.
import numpy as np
import cv2
import dropbox
import os
from glob import iglob
access_token = 'paste your access token here' #paste your access token in-between ''
client = dropbox.client.DropboxClient(access_token)
print 'linked account: ', client.account_info()
PATH = ''
cap = cv2.VideoCapture(0)
# Define the codec and create VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('C:\python27\output1.avi',fourcc, 20.0, (640,480))
#here output1.avi is the filename in which your video which is captured from webcam is stored. and it resides in C:\python27 as per the path is given.
while(cap.isOpened()):
ret, frame = cap.read()
if ret==True:
#frame = cv2.flip(frame,0) #if u want to flip your video
# write the (unflipped or flipped) frame
out.write(frame)
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
# Release everything if job is finished
cap.release()
out.release()
cv2.destroyAllWindows()
for filename in iglob(os.path.join(PATH, 'C:/Python27/output1.avi')):
print filename
try:
f = open(filename, 'rb')
response = client.put_file('/livevideo1.avi', f)
print "uploaded:", response
f.close()
#os.remove(filename)
except Exception, e:
print 'Error %s' % e
Here is the code for uploading existing video on your dropbox account using python in windows.
Hope this will help you.
# Include the Dropbox SDK
import dropbox
# Get your app key and secret from the Dropbox developer website
app_key = 'paste your app-key here'
app_secret = 'paste your app-secret here'
flow = dropbox.client.DropboxOAuth2FlowNoRedirect(app_key, app_secret)
# Have the user sign in and authorize this token
authorize_url = flow.start()
print '1. Go to: ' + authorize_url
print '2. Click "Allow" (you might have to log in first)'
print '3. Copy the authorization code.'
code = raw_input("Enter the authorization code here: ").strip()
# This will fail if the user enters an invalid authorization code
access_token, user_id = flow.finish(code)
client = dropbox.client.DropboxClient(access_token)
print 'linked account: ', client.account_info()
f = open('give full path of the video which u want to upload on your dropbox account(ex: C:\python27\examples\video.avi)', 'rb')
response = client.put_file('/video1.avi', f) #video1.avi is the name in which your video is shown on your dropbox account. You can give any name here.
print 'uploaded: ', response
folder_metadata = client.metadata('/')
print 'metadata: ', folder_metadata
f, metadata = client.get_file_and_metadata('/video1.avi')
out = open('video1.avi', 'wb')
out.write(f.read())
out.close()
print metadata
Now for uploading images, the same code will be used.
Only write your image file name which you want to upload for ex: image.jpg in place of video name . Also change the name of video1.avi and write name for image in which your uploaded image will be shown in your dropbox for ex:image1.jpg.