I need to access a Confluence page via python script and change all 'use_windows' strings to 'use_linux' strings
from sh import sed
from atlassian import Confluence
confluence = Confluence(
url='https://confluence.com/pages/viewpage.action?pageId=89967548'
username='user'
password='passs')
I can use want use this command inside of script:
sed(['-i', 's/use_windows/use_linux/'])
I can see you use the https://github.com/atlassian-api/atlassian-python-api lib. You do NOT need to specify direct URL to page, just Confluence SERVER URL (to core page): "https://confluence.com".
Here is the option:
from atlassian import Confluence
confluence = Confluence(
url='https://my-confluence-site.com',
username='admin',
password='admin')
page_id = 111111 # page id (or you can use space and title)
page_body = get_page_by_id(page_id ).get("body").get("storage").get("value")
page_body = page_body.replace("use_windows", "use_linux")
# set the new body for page
update_existing_page(page_id, title, page_body)
Related
I am trying to download some photos from Flickr. With My KEY and Secret, I am able to search and download using these lines of code
image_tag = 'seaside'
extras = ','.join(SIZES[0])
flickr = FlickrAPI(KEY, SECRET)
photos = flickr.walk(text=image_tag, # it will search by image title and image tags
extras=extras, # get the urls for each size we want
privacy_filter=1, # search only for public photos
per_page=50,
sort='relevance',
safe_search = 1 )
Using this I am able to acquire the url and the photo ID but I would like to download photostats too (likes, views), but I can't find an appropriate command that starting from the ID of the photo allows me to download the stats.
You can find what you are looking for exactly on Flickr web page, in the API's documentation:
https://www.flickr.com/services/api/flickr.stats.getPhotoStats.html
Calling the method:
flickr.stats.getPhotoStats
with arguments:
api_key, date, photo_id
You will receive what you look for in the following format:
<stats views="24" comments="4" favorites="1" />
Remember to generate before your authentication token, there is a link in this same page on how to generate it, if you still didn't.
I have a subscription to the site https://www.naturalgasintel.com/ for daily feeds of data that show up on their site directly as .txt files; their user login page being https://www.naturalgasintel.com/user/login/
For example a file for today's feed is given by the link https://naturalgasintel.com/ext/resources/Data-Feed/Daily-GPI/2019/01/20190104td.txt and shows up on the site like the picture below:
What I'd like to do is to log in using my user_email and user_password and scrape this data in the form of an Excel file.
When I use Twill to try and 'point' me to the data by first logging me into the site I use this code:
from email.mime.text import MIMEText
from subprocess import Popen, PIPE
import twill
from twill.commands import *
year= NOW[0:4]
month=NOW[5:7]
day=NOW[8:10]
date=(year+month+day)
path = "https://naturalgasintel.com/ext/resources/Data-Feed/Daily-GPI/"
end = "td.txt"
go("http://www.naturalgasintel.com/user/login")
fv("2", "user[email]", user_email)
fv("2", "user[password]", user_password)
fv("2", "commit", "Login")
datafilelocation = path + year + "/" + month + "/" + date + end
go(datafilelocation)
However, logging in from the user login page sends me to this referrer link when I go to the data's location.
https://www.naturalgasintel.com/user/login?referer=%2Fext%2Fresources%2FData-Feed%2FDaily-GPI%2F2019%2F01%2F20190104td.txt
Rather than:
https://naturalgasintel.com/ext/resources/Data-Feed/Daily-GPI/2019/01/20190104td.txt
I've tried using modules like requests as well to log in from the site and then access this data but whatever method I use sends me to the HTML source rather than the .txt data location itself.
I've posted my complete walk-through with the Python 2.7 module Twill which I attached a bounty to here:
Using Twill to grab .txt from login page Python
What would the best solution to being able to access these password protected files be?
If you have a compatible version of FireFox for this, then get the plugin javascript 0.0.1 by Chee and add the following to run on the page:
document.getElementById('user_email').value = "E-What";
document.getElementById('user_password').value = " ABC Password ";
Change the email and password as you like. It will load the page, then after that it will put in your username and password.
There are other ways to do this all by yourself with your own stand-alone process. You do not have to download other people's programs and try to learn them (beyond this little thing) if you change it this way.
I would have up voted this question.
I have a python script which is trying to export a confluence page as pdf and have tried several methods unsuccessfully:
1.WGET:
wget --ask-password --user xxxxxxxx -O out.pdf -q http://confluence.xxxx.com/spaces/flyingpdf/pdfpageexport.action?pageId=xxxxxxxx
This won't work because it just returns a login dialog rather than the actual pdf.
2.REMOTE API:
Using: https://developer.atlassian.com/confdev/deprecated-apis/confluence-xml-rpc-and-soap-apis/remote-confluence-methods#RemoteConfluenceMethods-Pages
There is an exportSpace method which works but I only want a single page, the getPage method doesn't export to pdf as far as I can tell. Also this is technically deprecated so Atlassian instead recommends:
3.REST API
Using: https://docs.atlassian.com/atlassian-confluence/REST/latest-server/
This doesn't seem to have an option to export a page as PDF
I would appreciate an answer that makes any of these methods work or if you have a completely different approach, I don't care as long as I can get the PDF of the page from a python script.
#This worked for me
#pip install atlassian-python-api
from atlassian import Confluence
#This creates connection object where you provide your confluence URL and credentials.
confluence = Confluence(
url='https://confluence.xxxxx.com/',
username='xxxxxxx',
password='yyyyyyy')
# If you know page_id of the page, you can get page id by going to "Page Information" menu tab and the page id will be visible in browser as viewinfo.action?pageId=244444444444. This will return a response having key:value pairs having page details.
page = confluence.get_page_by_id(page_id=<some_id>)
your_fname = "abc.pdf"
#A function to create pdf from byte-stream responce
def save_file(content):
file_pdf = open(your_fname, 'wb')
file_pdf.write(content)
file_pdf.close()
print("Completed")
#Get your confluence page as byte-stream
response = confluence.get_page_as_pdf(page['id'])
#Call function that will create pdf and save file using byte-stream response you received above.
save_file(content=response)
Based on the answer of Michael Pillai I want to amend that for the Confluence Cloud you must add the api_version='cloud' keyword:
confluence = Confluence(
url='https://confluence.xxxxx.com/',
username='xxxxxxx',
password='yyyyyyy',
api_version='cloud' # <<< important for the pdf export <<<<
)
content = confluence.get_page_as_pdf(page_id)
Copied from the official pdf export example.
You can integrate your script with Confluence Bob Swift CLI plugin. This plugin supports various type of exports.
Step 1: Install the plugin in both places the frontend and backend.
Step 2: Verify your installation by this command -
/location-of-plugin-installation-directory/.confluence.sh --action getServerInfo
Step 3: Use the command below to export your space
/location-of-plugin-installation-directory/.confluence.sh --action exportSpace --space "zconfluencecli" --file "target/output/export/exportSpacepdf.txt" --exportType "PDF"
Link to bob swift plugin
The newest docs use
confluence.export_page(page_id)
For anyone still looking for this info.
I'm trying to crawl Youtube to retrieve information about a group of users (approx. 200 people). I'm interested in looking for relationships between the users:
contacts
subscribers
subscriptions
what videos they commented on
etc
I've managed to get contact information with the following source:
import gdata.youtube
import gdata.youtube.service
from gdata.service import RequestError
from pub_author import KEY, NAME_REGEX
def get_details(name):
yt_service = gdata.youtube.service.YouTubeService()
yt_service.developer_key = KEY
contact_feed = yt_service.GetYouTubeContactFeed(username=name)
contacts = [ e.title.text for e in contact_feed.entry ]
return contacts
I can't seem the get the other bits of information I need. The reference guide says that I can grab the XML feed from http://gdata.youtube.com/feeds/api/users/username/subscriptions?v=2 (for some arbitrary user). However, if I try to get other users' subscriptions, I get the a 403 error with the following message:
User must be logged in to access these subscriptions.
If I use the gdata API:
sub_feed = yt_service.GetYouTubeSubscriptionFeed(username=name)
sub = [ e.title.text for e in contact_feed.entry ]
then I get the same error.
How can I get these subscriptions without logging in? It should be possible, as you can access this information without logging in to the Youtube web-site.
Also, there seems to be no feed for the subscribers of particular user. Is this information available through the API?
EDIT
So, it appears this can't be done through the API. I had to do this the quick and dirty way:
for f in `cat users.txt`; do wget "www.youtube.com/profile?user=$f&view=subscriptions" --output-document subscriptions/$f.html; done
Then use this script to get out the usernames from the downloaded HTML files:
"""Extract usernames from a Youtube profile using regex"""
import re
def main():
import sys
lines = open(sys.argv[1]).read().split('\n')
#
# The html files has two <a href="..."> tags for each user: once for an
# image thumbnail, and once for a text link.
#
users = set()
for l in lines:
match = re.search('<a href="/user/(?P<name>[^"]+)" onmousedown', l)
if match:
users.add(match.group('name'))
users = list(users)
users.sort()
print users
if __name__ == '__main__':
main()
In order to access a user's subscriptions feed without the user being logged in, the user must check the "Subscribe to a channel" checkbox under his Account Sharing settings.
Currently, there is no direct way to get a channel's subscribers through the gdata API. In fact, there has been an outstanding feature request for it that has remained open for over 3 years! See Retrieving a list of a user's subscribers?.
I have a python script running inside the Google App Engine with boto 1.9b that gets all keys inside a S3-Bucket. The output is formated as a HTML-Table.
bucket_instance = conn_s3.get_bucket(bucketname)
liste_keys = bucket_instance.get_all_keys()
table = '<table>'
for i in range(laenge_liste_keys):
table = table + '<tr><td>'+str(liste_keys[i].name)+</td></tr>'
table = '</table>'
How can I realize the key-names as links that enable the user to download the key via the browser?
Thanks in advance!
The solution is found.
generate_url(expires_in, method='GET', headers=None, query_auth=True, force_http=False)
This makes it easy to create a link for every key which is valid for x seconds.
The public URL for the file would be something like:
http://s3.amazonaws.com/bucket_name/key_name
So in your code, add links with their href attributes pointing to that URL.