I have the django application, which needs refresh some data. This data should be downloaded from my dropbox account (file name and path is the same each time). How can I implement this?
I start with using dropbox api, create application, etc - but this method has one big defect - it needs user go to the generated link and authorize to dropbox account. But I need automatic work, script should be executed by cron each day without userinteraction.
I think about using Selenium to open this link, enter login and password, confirm using application. But I also think this is hard way, should be another way:-)
Or maybe I can simply generate link to file one time and then use it every time I want to download file?
You could use the API and connect with pre-authorized access token which you authorized manually once (as opposed to having the user authorize their own account). You could then download the file from your account, but be sure not to revoke the access token, e.g. via https://www.dropbox.com/account/applications .
If you do just need to download files though, using a shared link may be easier:
https://www.dropbox.com/help/167/en
https://www.dropbox.com/help/201/en
They don't expire, but they can be revoked via https://www.dropbox.com/links .
Or if you prefer to use the Public folder, same idea:
https://www.dropbox.com/help/16/en
Related
Thanks for reading, I'm not asking for code snippets, but just more an overall architectural explanation of how the below could be achieved, ideally with best practices, would be very appreciated.
I have a situation where I allow users to upload files from their google drive that is later processed (essentially read) by a celery task. The current flow just authenticates the user through oauth2 and gets the file ID for the file. Then I just save a URL in the form of https://drive.google.com/uc?export=download&id=${file.id} into the database which is later read by the celery task and using the requests library, downloaded and read.
This works fine for unrestricted (publicly shared) files but when the files are set to have some restrictions (shared only to a group, to specific users, etc), it returns request forbidden 403s.
I'm currently following https://developers.google.com/drive/api/v3/manage-downloads guide and got a snippet working that downloads a restricted file. Essentially, it does a separate oauth2 flow initiated by the server and writes a token.pickle file that is later read to authenticate the file download request and voila, it succeeds in downloading.
The problem is tying the two flows together, specifically, how would / should I set this up so that the celery task is able to download the file sometime after oauth2 was completed?
I'm thinking I'd create the token.pickle file for a user somewhere that celery can read from and do the downloading. I'm not sure if there are any gotchas or security concerns with this.
Altogether, I'm using AWS, so I could put a token-for-user-123.pickle file in S3 from the app, save the bucket name to a DB record, and have the celery task query for that and read from that location? I suppose that would work, but I'm not sure of the security repercussions and even less sure of what would happen when the tokens expire, whenever that is. Files only process on a separate request, so it could happen seconds after the users authenticate and select a file or never.
One last thing, if there were a google URL I could hit with the tokens passed in in the form of something like https://drive.google.com/uc?export=download&id=${file.id}&token={token}&whatever-else={whatever-else} that can download restricted files, that would be amazing!
Thank you!
Dear stackoverflow community, i'am currently developing a plugin with Python which extract data (payload) about developers activity on repositories (push, pull requests) from bitbucket using webhooks. In my python script , i am trying to access to the patch and diffstat url's (Bitbucket API) content to extract data about files changes.
here example of the url's:
https://api.bitbucket.org/2.0/repositories/username/projectname/patch/cac0848613fea833316fcf7d87da9f80a5a2174e
https://api.bitbucket.org/2.0/repositories/username/projectname/diffstat/c6899a0a271db82790bf57de35dbbafb099417e4
But when the repository is private i cant access to the content of these web pages. If in the owner change private to public everything is okey but he can't change it for privacy so my question is how can access to these url's in my python script using the authentication? How can i authenticate via the script python knowing that i have write access on the repository?
thank you,
You probably need a token. From the bitbucket docs
Creating personal access tokens
To create a personal access token:
Go to Profile picture > Manage account > Personal access tokens.
Click Create a token.
Set the token name, permissions, and expiry.
I'm working on small python script (raspberry pi + Linux) that getting filename, as script argument, and upload it to Google drive.
In order to upload file to Google drive, I'm using this tutorial:
https://developers.google.com/drive/web/quickstart/quickstart-python
This script is basically working good, but, it's require manual authorization of the request - EACH time. This impossible when developing automated background task.
What I'm want to improve is to accept my application only once. From this time, all the file upload tasks will pass without security questions.
How to achieve this?
You want to follow server-side auth. Basically you store a refresh token that you receive the first time the user authorizes you, and you can use that to get new tokens without prompting the user.
See https://developers.google.com/drive/web/auth/web-server
I am trying to serve files securely (images in this case) to my users. I would like to do this using flask and preferably amazon s3 however I would be open to another cloud storage solution if required.
I have managed to get my flask static files like css and such on s3 however this is all non-secure. So everyone who has the link can open the static files. This is obviously not what I want for secure content. I can't seems to figure out how I can make a file available to just authenticated user that 'owns' the file.
For example: When I log into my dropbox account and copy a random file's download link. Then go over to anther computer and use this link it will denie me access. Even though I am still logged in and the download link is available to user on the latter pc.
Make the request to your Flask application, which will authenticate the user and then issue a redirect to the S3 object. The trick is that the redirect should be to a signed temporary URL that expires in a minute or so, so it can't be saved and used later or by others.
You can use boto.s3.key.generate_url function in your Flask app to create the temporary URL.
I would like for a user, without having to have an Amazon account, to be able to upload mutli-gigabyte files to an S3 bucket of mine.
How can I go about this? I want to enable a user to do this by giving them a key or perhaps through an upload form rather than making a bucket world-writeable obviously.
I'd prefer to use Python on my serverside, but the idea is that a user would need nothing more than their web browser or perhaps opening up their terminal and using built-in executables.
Any thoughts?
You are attempting to proxy the file thorough your python backend to S3, that too large files. Instead you can configure S3 to accept files from user directly (without proxying through your backend code).
It is explained here: Browser Uploads to S3 using HTML POST Forms. This way your server need not handle any upload load at all.
If you also want your users to use their elsewhere ID (google/FB etc) to achieve this workflow, that too is possible. They will be able to upload these files to a sub-folder (path) in your bucket without exposing other parts of your bucket. This is detailed here: Web Identity Federation with Mobile Applications. Though it says mobile, you can apply the same to webapps.
Having said all that, as #Ratan points out, large file uploads could break in between when you try from a browser and it cant retry "only the failed parts". This is where a dedicated app's need come in. Another option is to ask your users to keep the files in their Dropbox/BOX.com account and your server can read from there - these services already take care of large file upload with all retries etc using their apps.
This answer is relevant to .Net as language.
We had such requirement, where we had created an executable. The executable internally called a web method, which validated the app authenticated to upload files to AWS S3 or NOT.
You can do this using a web browser too, but I would not suggest this, if you are targeting big files.